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1.0 SUMMARY 


The flight management computer (FMC) control display unit (CDU) test was conducted to 
compare two types of input devices: a fixed legend (dedicated) key board and a programmable 
legend (multifunction) keyboard. The task used for comparison was operation of the flight 
management computer for the Boeing 737-300. The same tasks were performed by 12 pilots on 
the FMC control display unit configured with a programmable legend keyboard and with the 
currently used B737-300 dedicated keyboard. Flight simulator work activity levels and input 
task complexity were varied during each pilot session. Half of the pilots tested were previously 
familiar with the B737-300 dedicated keyboard CDU and half had no prior experience with it. 
The data collected included simulator flight parameters, keystroke times and sequences, and 
pilot questionnaire responses. A timeline analysis was also used for evaluation of the two 
keyboard concepts. 

For operation of the FMC, the number of keystrokes needed to complete each task on the 
multifunction keyboard (MFK) was generally more than that needed on the dedicated key- 
board (DK). The majority of the additional keystrokes are attributable to the double keystroke 
needed to enter alpha information. All pilots, familiar and unfamiliar, performed the tasks 
faster on the DK. The average time per keystroke, however, was found to be lower on the MFK 
than the DK for unfamiliar pilots. This shows a tendency for pilots equally familiar with both 
units (i.e., the unfamiliar pilots in this study) to perform keystrokes faster on the MFK. 

Several factors of interest in this study were not affected by the CDU type. Although flight 
activity had a greater effect on the average time per keystroke, there was no relative differ- 
ence in the performance on the two CDU types. No difference between CDUs was found in the 
pilot performance of the flying tasks. Also, the number of errors per keystroke was not signifi- 
cantly different on the MFK and the DK. 

The input time was found to increase from numeric to alpha to system command and line 
select for both CDU types. The type of CDU did not have a differential effect on input times, 
although it is possible that this interaction may have been masked. 

Pilot opinion was not favorable for some logic and task structure aspects of the MFK. The 
problem areas appear to be in the total number of steps required, and in the input of alpha 
information. Pilots perceived the workload to be higher on the MFK than on the DK. TLA 
results, however, show that for implementation of an advanced flight deck concept, the MFK 
performs as well as or better than the DK in terms of workload. 

Functions chosen for implementation on an MFK should be selected on the basis of task 
structure as compared to a conventional DK. In particular, the number of keystrokes on the 
MFK should not greatly exceed the keystrokes on the DK. Several methods of enhancing MFK 
operation in other applications include tailoring the keyboard to the flight phase, using the 
line select keys to call up input keyboards, providing a mix of dedicated and multifunction 
keys, and the use of artificial intelligence in conjunction with the MFK. 


1 



2.0 INTRODUCTION 


2.1 BACKGROUND 

A dedicated keyboard (DK) has keys, or switches, which always perform the same function. 
Activation of a given key will always give the same results. A multifunction keyboard (MFK) 
has keys that are capable of changing their function. The current function of a particular key 
is displayed on its programmable legend keyface. The MFK requires fewer keys than the 
dedicated keyboard to accomplish the same functions, and can incorporate several dedicated 
controls into one unit. 

If space requirements can be reduced by using a single multifunction input device in place of 
several dedicated devices, displays and controls can be located more optimally with respect to 
the pilot’s vision and reach envelopes. Multifunction controls also have the potential for reduc- 
ing crew workload and managing the information flow by restricting the information pre- 
sented to that which is relevant to the current task. Information not relevant can be made 
available on request so that it can be accessed if needed. 

Before implementation of an MFK on the flight deck, the effect it will have on the pilot must 
be evaluated. “Research must be conducted to determine the optimal method of implementing 
the multifunction control and identify the data display and switching requirements. Once a 
candidate multifunction control is designed, it is necessary to evaluate it in comparison with 
the operator’s conventional controls. Design and evaluation must continue until the use of 
multifunction controls increases the control capability and information available to the opera- 
tor and makes the completion of the required tasks more effective” (ref. 1). One criterion of 
effectiveness is the time it takes an operator to perform a task. “The time cost required for 
setting each function using the multipurpose device must be less than or equal to the time cost 
required for setting the function using a dedicated control” (ref. 2). 

Other criteria are of importance when replacing conventional controls with a multifunction 
unit. The overall workload imposed by the unit must be assessed and compared to the existing 
workload condition. Pilot acceptance must also be considered. It is unlikely that a control will 
be placed on the flight deck over the strong objections of pilots, regardless of improvements 
theoretically possible. 


2.2 PURPOSE 

Previous research has established some guidelines for MFK design (refs. 3 and 4), and has 
made recommendations regarding the MFK control logic (refs. 3, 5, and 6). The purpose of this 
study was to investigate some basic questions a flight deck designer must ask when deciding 
whether or not to implement a multifunction control. This study was designed to examine the 
differences in pilot performance and acceptance between the MFK and the DK when used for 
input to a flight management system. Several factors were of interest. The input type (alpha 
versus numeric versus system command versus line select) was thought to be a source of 
variance between the MFK and the DK. A second factor was pilot familiarity with dedicated 
keyboards (i.e., how does the fact that a pilot is already acquainted with the dedicated key- 
board control affect the performance on and acceptance of the same unit configured with a 
multifunction keyboard?) In relation to the task environment, it was desired to study the 
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differences due to the pilot activity demanded by the flight task. The effect of the complexity 
of the task was also of interest. Finally, a comparison of the . workload required to operate an 
MFK control and a DK control and an analysis of error rates was desired. 

The flight management computer CDU test compared two types of input devices: a dedicated 
keyboard and a multifunction keyboard. The task that was used for evaluation purposes was 
the operation of the flight management computer (FMC) for the Boeing 737-300. The FMC 
control display unit (CDU) is the interface between the crew and the FMC. Pilots were asked 
to perform the same tasks on the FMC CDU configured with a multifunction keyboard and 
with the currently used B737-300 dedicated keyboard. The display contents were the same for 
both units; only the keyboard type changed. 

Twelve pilots participated in the experiment. All performed a number of tasks on the dedi- 
cated keyboard (DK) CDU and on the multifunction keyboard (MFK) CDU. Pilots who were 
unfamiliar with both units and pilots who had extensive experience using the B737-300 FMC 
participated in the experiment. Some tasks were conducted while flying the simulator; flight 
simulator work activity was varied between tasks. The complexity of the CDU task was also 
varied. 

Performance measures were taken on the flight parameters and on keystroke times and se- 
quences. Subjective evaluations of the pilots were recorded via questionnaires. An analytical 
study was also performed to compare the work load differences in the operation of the two 
CDU types. 
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3.0 METHOD 


3.1 SUBJECTS 

Twelve pilots participated in this experiment. Six of the pilots were familiar with the flight 
management computer operation. Five of these six were Boeing Flight Crew Training pilots. 
The sixth was a military pilot and had participated in FMC development as a Boeing engineer. 

All of the familiar pilots had extensive experience with FMC operation. The five Flight Crew 
Training pilots train commercial airline pilots on the FMC. Because they use the FMC so 
often in training situations, they have achieved a greater level of familiarity with FMC opera- 
tion than most airline pilots achieve under normal circumstances. The sixth familiar pilot, 
because of his involvement with FMC development, was also quite familiar with its operation. 
These pilots were selected for participation in this study because it was thought that their 
level of familiarity with the FMC would cause them to be sensitive to differences between the 
dedicated and multifunction keyboards. 

Six pilots unfamiliar with FMC operation also participated in the test. Five were currently 
pilots flying for commercial airlines. The sixth was an air line pilot who had recently retired 
with over 25,000 hours of flight time. None had experience operating the FMC or had more 
than a cursory knowledge of the keyboard layout. 

3.2 EXPERIMENTAL DESIGN 

Four factors were investigated in this study: pilot familiarity, CDU type, flight simulator 
activity, and input task complexity. Pilot familiarity had two levels (familiar versus unfamil- 
iar), and was the only factor not to have repeated measures. CDU type (two levels) included 
the dedicated keyboard CDU currently used to operate the FMC on the 737-300 and a multi- 
function keyboard CDU. 

Flight simulator activity occurred at three levels, or flight phases. These were defined as on 
the ground (none or low activity), climbing with control wheel steering (medium activity), and 
hand flying prior to top of descent during moderate turbulence (high activity). 

The fourth factor was task complexity (two levels). A high complexity task required more steps 
to complete and generally contained more system command entry or changing to different 
“pages” in the FMC data base, as compared to a low complexity task. 

The flight activity and task complexity factors together defined the operations used during the 
test. For each of the three flight activity levels, two operations were performed (one low com- 
plexity and one high complexity); resulting in a total of six test scenarios. See Table 1 for a 
description of the scenarios. 

The experimental design was a factorial experiment with repeated measures on all but one 
factor. The experimental design is displayed graphically in Figure 1. Table 2 presents a sum- 
mary of the calculations for mean square and F ratio used in the analysis of variance of the 
objective performance data. The degrees of freedom for the F ratios are the denominators of 
the mean square terms. The significance level was defined at p<.05. 
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Table 1. Scenario Description 


Scenario Flight Task Number of Keystrokes 

Number Operation Activity Complexity Alpha Numeric System/LS Total 


1 

Flight Plan Entry 

Low 

High 

2 

Performance 

Initialization 

Low 

Low 

3 

Route 

Modification 

Medium 

High 

4 

Altitiude 

Restriction Entry 

Medium 

Low 

5 

Approach Setup 

High 

Low 

6 

Time/Distance to 
Crossing Radial 

High 

High 


Total (Scenarios 1 Through 6) 


MFK 

33 

2 

18 

53 

DK 

17 

2 

17 

36 

MFK 

0 

21 

12 

33 

DK 

0 

19 

11 

30 

MFK 

28 

0 

14 

42 

DK 

14 

0 

12 

26 

MFK 

2 

9 

8 

19 

DK 

1 

8 

4 

13 

MFK 

0 

0 

8 

8 

DK 

0 

0 

9 

9 

MFK 

6 

5 

12 

23 

DK 

3 

4 

9 

16 

MFK: 69 

37 

72 

178 

DK 

35 

33 

62 

130 


Whenever task performance is measured under several different treatment conditions over an 
extended period of time, learning and/or fatigue may affect performance on later trials. An 
appropriate counterbalancing scheme was designed to prevent carryover effects from differen- 
tially affecting the performance measures for the different treatment conditions. Each pilot 
performed all the trials on a given CDU before moving on to the next. This is done to facilitate 


Pilot Familiarity 


Familiar Pilots 


Unfamiliar Pilots 


CDU 

Type 



CDU 

Type 



Figure 1. Experimental Design 
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Table 2. Summary Calculations for Analysis of Variance, Ax (BxCxDxS) 


SOURCE 

MEAN SQUARE 

F 

A 

$$ A /(a-l) 

ms a /ms s/a 

Between Subject 
Error (S/A) 

SS S/A^ s_a ^ 


B 

SS B /(b-l) 

MSg/MSgs/A 

AB 

SS A 6 /(a-l)(b-l) 

ms ab /ms bs/a 

Within Subject 
Error (BS/A) 

SS BS/A^ t> ‘ 1 ^ s " a ^ 


C 

SS c /(c-l) 

ms c /ms cs/a 

AC 

SS AC /(a-l)(c-l) 

ms ac /ms cs/a 

Within Subject 
Error (CS/A) 

SS CS/A^ C-1 ^ s “ a ^ 


D 

SS D /(d-l) 

ms i/ ms ds/a 

AD 

SS AD /(a-l)(d-l) 

ms ad /ms ds/a 

Within Subject 
Error (DS/A) 

SS DS/A^ d_1 ^ s_a ^ 


BC 

SS BC /(b-l)(c-l) 

ms bc 7 MSB bcs/a 

ABC 

ss ABC /( a - 1 H b - 1 ) (c - 1 ) 

ms abc /ms bcs/a 

Within Subject 
Error (BCS/A) 

SS BCS/A /(b - 1){c - 1)(s - a) 


BD 

SS BD /(b-l)(d-l) 

ms bd /ms bds/a 

ABD 

SS ABD /(a-l)(b-l)(d-l) 

ms abd /ms bds/a 

Within Subject 
Error (BDS/A) 

SS BDS/A /(b_1)(d ' 1)(s_a) 


CD 

SS C 0 /(c-l)(d-l) 

ms cd /ms cds/a 

ACD 

SS ACD /(a-l)(c-l)(d-l) 

ms acd /ms cds/a 

Within Subject 
Error (CDS/A) 

SS CDS/A /(c ' 1)(d - 1)(S - a) 


BCD 

SS BCD /(b-l)(c-l)(d-l) 

ms bcd /ms bcds/a 

ABCD 

SS ABCD 7 ( a_1 ) ( b_1 ) ( 0-1 ) ( d_1 } 

ms abcd /ms bcds/a 

Within Subject 
Error (BCDS/A) 

SS BCDS/A /(b ' 1)(c ' 1)(d " 1)(S_a) 
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Figure 2. Test Setup 


training and reduce the potential for confusion between CDU operations. The order of receiv- 
ing the different CDU types and the order of task assignment within CDU type were ran- 
domly assigned to prevent order bias from confounding the results. 

3.3 EQUIPMENT 


3.3.1 Hardware 

The test setup is shown in in Figure 2. A test-bench flight simulator is shown in the figure. 
The MFK CDU was physically larger than the DK CDU. The test-bench center console could 
accommodate only the DK CDU, and modifications to the console were not considered accept- 
able. A new console was constructed, therefore, which accommodated both CDU types. It was 
located to the right of the test bench, and the pilots flew from the right-hand seat. Thus, the 
relationship of the CDU to the pilot’s control and displays was configured as if the pilot were 
flying from the left-hand seat, except that the throttles were located on the pilot’s right. This 
was not expected to have an effect on the test results, as little throttle input was required 
during the flying tasks. 

CDUs were swapped halfway through the test, so that both CDU types would be operated from 
the same location. The console was positioned so that pilots, when adjusted to the eye refer- 
ence position, could easily see and reach the CDU. A chair was placed to the right of the CDU 
console for the instructor/observer pilot. 
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The console to the left of the test bench controlled the simulator. The initial conditions for each 
task were set at this location, and the flight parameter data collection was controlled here 
also. The principle investigator operated the test console and was able to observe the subject 
pilot from this location. 

The two types of CDUs used in the test are shown in Figures 3 and 4. The dedicated keyboard 
CDU is currently used as the pilot interface to the FMC on the B737-300 and is very similar to 
the B757 and B767 FMC CDUs. The multifunction keyboard CDU was made up of a Litton 
display and a Microswitch keyboard. The available display area on the MFK was longer than 
that on the DK, but the area of the display used (programmed) for the test was virtually 
identical. 

The primary character size was 0.165-in high for data on the DK display, and 0.172-in high on 
the MFK display. 

The content of the displays on the MFK CDU and the DK CDU were identical. This was done 
to eliminate any differences in pilot performance due to the displays so that a direct compari- 
son could be made between keyboard types. 

The MFK contained 15 programmable switches. Entry of an alpha character into the scratch 
pad required the pilot to make two switch hits. The keyboard for alpha entry was configured 
as shown in Figure 5, with three letters per key. Tb enter the letter H, the pilot selected the 
key GHI. The keyboard would reconfigure to offer the choice of G, H, or I, as shown in Figure 
6. The three letters in the reduced alpha configuration always appeared in the same key row 
as the root key. For example the three keys S,T, and U appeared in the third row of keys after 
the STU key was selected from the full alpha keyboard configuration. The double hit require- 
ment is largely responsible for the difference between the MFK and the DK in the total 
number of switch hits required to complete a scenario, although some differences do occur in 
the number of numeric and system command switch hits required. Table 1 lists the number of 
keystrokes required to complete each scenario, and breaks down the total number into three 
categories: alpha, numeric, and the combination of system command plus line select. 

3.3.2 Software 

The control logic has been called “one of the most critical design considerations” for a multi- 
function control (ref. 3). Two types of control logic for use on a multifunction control have 
received the most attention from investigators: tailored logic and branching logic. Branching 
logic follows a logic tree and does not change during a flight. Tailored logic provides access to 
functions frequently used; the functions on the switches are dependent on the flight mode. 
Tailored logic serves to reduce the number of key hits required to reach the desired keyboard 
configuration. Studies have shown that pilot performance on a multifunction control is en- 
hanced when tailored logic is used instead of or in conjunction with branching logic (refs. 3, 5, 
and 6). 

In this test, the concept of tailored logic was applicable to a limited extent. Keyboard configu- 
rations that would allow input of alpha or numeric information were only one or two levels 
down from the top level configuration. However, tailored logic was used when possible. An 
example is selection of the DEP/ARR (departure/arrival) switch on the top level keyboard. On 
the ground, the pilot would then have to indicate w'hether departure or arrival pages were to 
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Figure 5. Full Alpha Configuration 
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Figure 6. Reduced Alpha Configuration 



be displayed. In the air, the departure pages are not needed, and selection of the DEP/ARR 
switch automatically brings up the arrival pages. This was incorporated into Scenario 5. 
Other efforts were made to “tailor” keyboard configurations, independent of flight phase. For 
example, if the majority of information to be entered onto a display page was alpha rather 
than numeric, the keyboard one step down from the top level keyboard would automatically 
provide the alpha configuration. The numeric keyboard couid be accessed if needed by select- 
ing the “number” switch on the alpha keyboard. 

3.4 PERFORMANCE MEASURES 

There were two types of objective performance measures taken during the test: flight parame- 
ter data and keystroke data. Flight parameters were recorded by the mainframe computer 
which ran the simulated airplane. The parameters recorded were altitude, airspeed, flightpath 
angle, and magnetic heading. Keystroke sequences and times were monitored and printed out. 

In addition to the objective measures, subjective data was collected using questionnaires. 
Three questionnaires were administered to each pilot. A postflight questionnaire was given to 
the pilots after operation of the DK and again after operation of the MFK to gather pilots’ 
immediate impressions. These postflight questionnaires asked pilots to rate the CDU just used 
on specific characteristics and operational aspects. A third questionnaire, the program ques- 
tionnaire was given to the pilots upon completion of the test. This questionnaire included 
comparisons between the two CDU types. Appendix A contains the questionnaires with a 
summary of all pilots’ responses. 


3.5 PROCEDURE 

One pilot was run per session for a total of 12 sessions. Each pilot was sent a copy of a training 
manual approximately one week prior to his scheduled session. A copy of the training manual 
is contained in Appendix B. It describes the test purpose, serves as an introduction to both the 
DK and MFK CDUs, and describes the test conditions and procedures. 

On arrival at the test location, the pilot was given a brief verbal introduction to the test 
purpose, setup, and procedure. A period of familiarization with the first CDU (either the DK 
or the MFK— the order of the test conditions were randomized over all subjects) was conducted 
using training scenarios. The six training scenarios were functionally identical to the sce- 
narios used during the test, but contained different alphanumeric data so that pilots could not 
memorize the keystroke sequences. The six training and six test scenarios are listed in Appen- 
dix C. The instructor pilot presented the navigation maps and explained what would occur in 
each scenario. The next scenario was introduced only after the pilot was able to complete the 
current training scenario with no errors. 

After all six training scenarios were completed, the pilot was adjusted to the eye reference 
position and allowed to fly the simulator to familiarize himself with the feel of the simulated 
airplane in both the control wheel steering (CWS) and turbulence conditions. Also, a training 
scenario was performed on the CDU while flying. After the pilot indicated he was ready to 
begin the test, the six test scenarios were performed on the first CDU. 

The test scenarios were randomly presented. The pilot was told verbally which scenario and 
corresponding flight phase would be done next. Flying tasks for Scenarios 3 through 6 were 
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presented verbally to the pilot before the simulator was active, and are listed below each 
scenario in Appendix C. At the signal from the instructor pilot, the principle investigator 
started the simulation and data recording apparatus. The pilot was instructed to begin the 
CDU operation only after he felt comfortable with the flight task (for the four out of six 
scenarios in the air). At the end of the CDU operation, the simulation and data recording was 
terminated, and the equipment was reset for the next scenario condition. 

The postflight questionnaire on the first CDU was given to the pilot after completion of all six 
scenarios. At this time, the CDUs were changed out. A familiarization period was then con- 
ducted with the second CDU, and the same six scenarios were performed on the second CDU. 
The postflight questionnaire for the second unit was given to the pilot. When this was com- 
pleted, the program questionnaire was given to the pilot to complete. Sessions lasted from 2 to 
4 hours; pilots familiar with FMC operation generally took less time as they required less 
training time. 
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4.0 RESULTS 


4.1 PERFORMANCE DATA ANALYSIS 
4.1.1 Total Time to Complete Tasks 

The total time to complete the tasks are analyzed in Table 3. All of the main effects tested 
significant at the p < .05 level. Pilots familiar with FMC operation performed the tasks faster 
than unfamiliar pilots. Flight activity was also significant. Total time to complete the tasks 
decreased with increasing simulator activity. This result is not meaningful, however, due to 
the different numbers of keystrokes required at the different activity levels. The two tasks 
that comprise the low activity level total 152 keystrokes, the medium activity level required 
100 keystrokes, and the high activity level required 56 keystrokes. 

Pilots performed tasks faster on the DK than on the MFK. As with the flight activity effect, 
there was a difference in the total number of keystrokes required on the MFK as compared to 
the DK. The MFK required 178 keystrokes to complete all six tasks, while the DK required 
130. This difference is mostly caused by the fact that the MFK requires two switch hits for an 
alpha entry, while the DK requires one switch hit. 

Figure 7 shows the interaction between pilot familiarity and task complexity. The unfamiliar 
pilots took a proportionally greater time on the high task complexity versus the low task 
complexity conditions as compared to the familiar pilots. 


Time 

(Seconds) 



Figure 7. Pilot Familiarity x Task Complexity (Total Time) 
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Table 3. Summary of Analysis of Variance From Total Time to Complete Tasks in Seconds 
(by Pilot Familiarity, Flight Activity, Task Complexity, and CDU Type) 


SOURCE OF VARIANCE 

DF 

F 

Main Effects: 
Pilot familiarity 

1 

22.8* 

Flight activity 

2 

25.6* 

Task complexity 

1 

335.5* 

CDU type 

1 

15.9* 

Interacti ons : 
Complexity X CDU type 

1 

46.4* 

Pilot familiarty X flight activity 

2 

1.6 

Pilot familiarity X Task complexity 

1 

14.2* 

Pilot familiarity X CDU type 

1 

3.4 

Flight activity X Task complexity 

2 

3.U 

Flight activity X CDU type 

2 

2.8 

Flight activity X Task complexity 
X CDU type 

2 

4.8* 

Pilot familiarity X Task complexity 
X Flight activity 

2 

2.0 

Pilot familiarity X Flight activity 
X CDU type 

2 

.2 

Pilot familiarity X Task complexity 
X CDU type 

1 

.9 

Pilot familiarity X Flight activity 
X Task complexity X CDU type 

2 

3.3 

* probability < ,0b 

Cell means and effects (in seconds): 

Pilot familiarity 

Familiar Unfamiliar 

62.9 95.6 

Pilots familiar with FMC operation 

performed the tasks 

faster . 

Flight activity 

Low Medium High 

94.2 78.9 64.6 




16 





Table 3. Summary of Analysis of Variance From Total Time to Complete Tasks in Seconds 
(by Pilot Familiarity, Flight Activity, Task Complexity, and CDU Type) (Concluded) 


Total time to complete task decreased with increasing flight activity. 

Task complexity 

Low High 

54.4 104.1 

Tasks with high complexity took more time to complete. 

CDU type 


MFK 

7S7T6 


DK 

7TJ.8 


Pilots performed tasks faster on the dedicated keyboard CDU than on the 
multifunction keyboard CDU. 

Pilot familiarity X Task complexity 

Familiar Unfamiliar 

Low 43.1 65.6 

Hi gh 82.6 125.6 

Unfamiliar pilots took a relatively longer time on the high task com- 
plexity conditions as compared to the familiar pilots (see Figure 7). 

Task complexity X CDU type 

Low Hi gh 

MFK 56.6 118.7 

T3T" 52.2 89.4 

The high task complexity conditions on the MFK show a di sproporti onately 
long time for task completion. This relationship is shown in Figure 8. 


Flight activity X Task complexity X CDU type 

Low Medium 

Low Hi gh Low Hi gh 

MFK "5571 14176 30 T2173 

W 67.5 98.4 40.1 94.3 


High 

Low High 
7ZFDT7 937J 
49. U 7b. 6 


The low task complexity/low flight activity condition shows the MFK total 
time just slightly higher than the DK. The low task complexity high flight 
activity condition shows the MFK total time as less than the DK's. See Figure 
9 for a graphic representation of this relationship. 






Time (sec) 



Task Complexity 

Figure 8. Task Complexity x CDU Type ( Total Time) 


The significant interaction between task complexity and CDU type is depicted in Figure 8. 
The high task complexity condition on the MFK took disproportionately longer than the oth- 
ers. Again, this result is not meaningful because there is a difference in the number of key- 
strokes required for the tasks. Because the low complexity conditions were mostly system 
command entry, there was little difference in the keystrokes required on the two CDU types 
(60 for the MFK and 52 for the DK). But because the high complexity conditions contained 
more alpha entries, the difference between the MFK and DK becomes greater (118 and 98, 
respectively). 

The interaction between flight activity and task complexity and CDU type, pictured in Figure 
9 shows the low task complexity/low flight activity condition (i.e., task 2) almost equal on the 
MFK and the DK. The number of keystrokes required for Task 2 is 33 on the MFK and 30 on 
the DK. Figure 9 also shows that the low task complexity/high flight activity condition (task 
5) took less time on the MFK than on the DK. This was the only scenario that required fewer 
keystrokes on the MFK than on the DK (8 versus 9, respectively). 

4.1.2 Average Time per Keystroke 

The average time per keystroke for a given task was calculated by dividing the total time for a 
given task by the total number of switch hits (including errors). This was done to eliminate 
the differences between the two CDU types due to the number of keystrokes, as the multifunc- 
tion keyboard generally required more keystrokes for a given task than the dedicated key- 
board. The significant results are presented below; see Table 4 for a summary of the analysis 
of variance. 


18 



Time (sec) 



Task Complexity: Low High Low High Low High 

- v ■" > ■ ■■ V S V ■ ■■■-" 
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Figure 9. Flight Activity x Task Complexity x CDU Type (Total Time) 

Pilots familiar with FMC operation had a overall lower average keying time than pilots unfa- 
miliar with FMC operation. Pilots familiar with FMC operation had a lower average keying 
time on the dedicated keyboard versus the multifunction keyboard, while for pilots unfamiliar 
with FMC operation, the converse was true. This interaction is shown in Figure 10. 

The type of CDU (multifunction keyboard versus dedicated keyboard) did not significantly 
affect the average time per keystroke for a given task. Although the total time to complete a 
task may be greater on the MFK than on the DK due to the greater number of keystrokes 
required on the MFK (as discussed above), there is no significant difference in terms of aver- 
age keying times. 

The amount of flight simulator activity occurring during a task significantly affected average 
time per keystroke. The higher the activity, the more time taken per average keystroke. 

Unfamiliar pilots had relatively higher average keying times during the high flight activity 
condition (see fig. 11). The significant interaction between task complexity and flight activity 
shows a significantly higher average keystroke time for the high activity, low task complexity 
combination (see fig. 12). This task (task 5) was comprised entirely of system command entry. 
As will be seen in Section 4.1.3, pilots made system command inputs slower than alpha or 
numeric inputs. 

The significant four-way interaction is shown in Figure 13. It can be seen that the unfamiliar 
pilots performed better on the DK than on the MFK only for one task. This is due to the fact 
that the MFK required twice as many system command/line select switch hits for that task 
than did the DK. The discussion below shows that this would result in a longer average time 
per keystroke. 
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Table 4. Summary of Analysis of Variance for Average Sec/Keystroke (by Pilot Familiarity, 
Flight Activity, Task Complexity, and CDU Type) 


SOURCE OF VARIANCE DF F 


Main Effects: 

Pilot familiarity 1 16.6* 

Flight activity 2 49.1* 

Task complexity 1 .1 

CDU type 1 . 1 

Interactions : 

Pilot familiarity X CDU type 1 6.6* 

Pilot familiarity X Flight activity 2 4.2* 

Flight activity X Task complexity 2 23.8* 

Pilot familiarity X Task complexity 1 1.8 

Flight activity X CDU type 2 .1 

Task complexity X CDU type 1 .4 

Pilot famil iarity X Flight activity 

X Task complexity 2 U 

Pilot familiarity X Flight activity 

X CDU type 2 l.U 

Pilot familiarity X Task complexity 

X CDU type 1 l.U 

Task complexity X Flight activity 

X CDU type 2 2.1 

Pilot familiarity X Flight activity 

X Task complexity X CDU type 2 3.9* 


* probability < .05 

Cell means and effects (in seconds): 

Pilot familiarity 


Familiar Unfamiliar 
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Table 4. Summary of Analysis of Variance for Average Sec/Keystroke (by Pilot Familiarity, 
Flight Activity, Task Complexity, and CDU Type) (Concluded) 


Flight activity 


Medium 

2.89 


Average keying times increased as flight simulator activity increased 
Pilot Familarity X CDU type 


Famil i ar 2.71 2.43 

Unfami 1 i'ar 3.56 4.04 

Familiar pilots performed faster on the DK versus the MFK. For unfam- 
iliar pilots, the converse was true. Figure 10 shows this relationship. 


Pilots familiarity X Flight activity 


Fami liar 
Unfami 1 iar 


Medium 

2.49 

3.29 


High 

TT38 


Average keying times for unfamiliar pilots at the high activity level 
were relatively greater than for familiar pilots. See Figure 11 for a graphic 
representation of this relationship. 

Task complexity X Flight activity 


Low 

mi 


Medium 

2.10 

3.08 


The low task complexity/high activity combination had high average keying 
times. See Figure 12 for a graphic representation. 

Pilot familiarity X Flight activity X Task complexity X CDU type 


Familiar 


Unfamiliar 


Low Medium High Low Medium H i gh 

Low Hi gh Low HTgh Low Hi gh Low High Low~ FTTgh Low * TTi gh 

MFK 1.83 2.15 2.43 2.68 4.25 2.93 2.25 2.95 3.33 2.98 5.15 4.73 

TjT" 1.62 1.80 2.18 2.65 3.58 2.77 2.78 3.33 2.83 4.UU 6.bb 4.7b 

Unfamiliar pilots performed better (faster average keying times) on the DK 
than on the MFK for the low task complexity, medium flight activity condition 
(task 4). This is the reverse of the results for all other tasks. Figure 13 
shows the interaction. Also, familiar pilots performed as well on the MFK as 
on the DK during the high task complexity, medium flight activity condition 
(task 3). 
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Figure 10. Pilot Familiarity x CDU Type (Average Keying Times) 


Seconds 

Keystroke 



Figure 11. Pilot Familiarity x Flight Activity (Average Keying Times) 



Seconds 

Keystroke 



Right Activity 

Figure 12. Flight Activity x Task Complexity (Average Keying Times) 


Seconds 

Keystroke 



Task Complexity Low High 
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Figure 13. Pilot Familiarity x Flight Activity x Task Complexity x CDU Type (Average Keying 

Times) 
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4.1.3 Input Type 


A single task was analyzed to examine the differing types of key inputs. The types of inputs 
were categorized as alpha, numeric, system command, and line select. Thsk 6 was chosen 
because it has a good mix of input types; no one input type predominated. Task 6 was the high 
flight activity, high complexity condition and required a total of 23 keystrokes on the MFK 
and 16 on the DK. The analysis was obtained by computing each pilots’ average time per 
input for each key type. Alpha input times for the multifunction keyboard were defined as the 
total time for the two keystrokes required for an alpha input. Table 5 summarizes the analysis 
of variance. 

Pilots familiar with FMC operation were faster than pilots unfamiliar with FMC operation. 
This result was also obtained in the analysis of total task time and of average keystroke time 
discussed above. 

Average times were significantly different between input types. Average times increased in 
the following order: numeric, alpha, system command, line select. Additional analysis re- 
vealed that the system command and line select times were not significantly different from 
each other. As in the analysis of average keying times above, this analysis revealed no signifi- 
cant difference in the type of CDU (multifunction keyboard versus dedicated keyboard) being 
used to perform Task 6. 

Further analysis was performed on the two alpha keystrokes comprising one alpha input for 
the multifunction keyboard. Because two keystrokes are required to enter an alpha character, 
it was desired to break the total alpha times into the first and second alpha keystrokes for 
Task 6. See Table 6 for the analysis of variance. 

The second keystroke for entering an alpha was performed significantly faster than the first. 
The data show that the second keystroke cell mean was less than half of the first keystroke 
cell mean. This is to be expected, as the second keystroke was a choice of 3 letters as opposed 
to 26. 

4.1.4 Keystroke Error Analysis 

A true error analysis could not be done for this test. Because the multifunction CDU ran a 
canned program, immediate feedback was provided when an error occurred. An incorrect key- 
stroke on the multifunction keyboard resulted in a question mark in the scratch pad, which 
the correct keystroke would erase. In contrast, operation of the dedicated keyboard did not 
provide this type of feedback and required the proper steps to erase or negate the erroneous 
key stroke(s). Therefore, for analysis purposes, “error sequences” were counted rather than 
single incorrect keystrokes. One error sequence is defined as all incorrect keystrokes (and 
correction keystrokes for the dedicated keyboard) in a single sequence. 

Table 7 summarizes the analysis of variance. It shows that significantly more error sequences 
occurred on the multifunction keyboard than on the dedicated keyboard. 

The MFK required more keystrokes to accomplish the same tasks than did the DK, thus 
allowing greater opportunity for errors. A second analysis of variance was done on the number 
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Table 5. Summary of Analysis of Variance for Average Sec/Input, Task 6 Only (by Pilot 
Familiarity, CDU Type, and Keytype) 


SOURCE OF VARIANCE 

DF 

F 

Main effects: 



Pilot familiarity 

1 

13.6* 

CDU type 

1 

1.2 

Keytype 

3 

21.5* 

Interact! - ons : 



Pilot familiarity X CDU type 

1 

.6 

Pilot familiarity X Keytype 

3 

1.1 

CDU type X Keytype 

3 

2.7 

Pilot familiarity X CDU type X Keytype 

3 

.4 


* probabi 1 ity < .05 

Cell means and effects (in seconds): 

Pilot familiarity 

Familiar Unfami liar 

3.4 5.31 


Pilots familiar with FMC operation were faster than unfamiliar pilots. 


Keytype 





Numeri c 

A1 pha 

System command 

Line select 

1.87 

3.59 

5.41 

5.82 


Average input time increase from numeric 
line select (system command and line select 
different from each other). 

to alpha to 
cell means 

system command and 
not significantly 
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Table 6. Summary of Analysis of Variance Average Sec/Keystroke, Task 6 MFK Alpha Only 
(by Pilot Familiarity and Alpha Keytype) 


SOURCE OF VARIANCE DF F 


Main effects: 

Pilot familiarity 
Alpha keytype 

Interacti ons : 

Pilot familiarity X Keytype 

* probability < .05 
Cell means and effects: 

A1 pha keytype 

A1 phal A1 pha2 

2.64 1.20 

The second keystroke for entering alpha information was performed faster 
than the f i rst . 


1 4.2 

1 27.0* 

1 1.9 


of errors per keystroke (the number of errors for a given task divided by the number of key- 
strokes to complete the task with no mistakes). This analysis showed no significant difference 
between the two CDU types. 

Flight Performance Data 

Tasks 1 and 2 occurred while on the ground and therefore had no flight parameters associated 
with them. Because pilots were instructed to maintain constant airspeed in Tasks 3 and 4, this 
flight parameter was chosen for analysis. Similarly, altitude was the parameter of interest for 
Tasks 5 and 6, as pilots were instructed to maintain constant altitude for those tasks. 

An attempt was made to subtract out the RMS errors of the time period before CDU operation 
commenced. This would have resulted in an RMS delta, and would have allowed an analysis 
on only the effect of using the two CDU types. However, in some cases pilot performance as 
defined by RMS error improved after CDU operation began. The log of the RMS error was 
used for the analysis of variance, as this generally results in a more normal distribution. 
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Table 7. Summary of Analysis of Variance for Error Sequences (by Pilot Familiarity and CDU 
Type) 


SOURCE OF VARIANCE 

DF 

F 

Main effects: 
Pilot familiarity 

1 

4.1 

CDU type 

1 

lb. 4* 

Interacti on$ : 

Pilot familiarity X CDU type 

1 

u 

* probabil ity < .05 



Cell means and effects: 

CDU type 

MFK DK 

4.8 2.7 

More error sequences occurred on 

the MFK than on the DK. 



Neither log RMS error for airspeed (tasks 3 and 4) or log RMS error for altitude (tasks 5 and 6) 
was significantly effected by pilot familiarity, task complexity, or CDU type, 

4.2 SUBJECTIVE EVALUATION 

The results of the pilots subjective evaluation is a compilation of the ratings and comments 
from three questionnaires: the Multifunction Keyboard CDU Postflight Questionnaire, the 
Dedicated Keyboard CDU Postflight Questionnaire, and the Program Questionnaire, which 
contained comparison questions regarding the two CDU types. Appendix A contains the ques- 
tionnaire results. The Kolmogorov-Smirnov test was the nonparametric statistic used in test- 
ing the significance of the questionnaire responses. The null hypothesis for the 
Kolmogorov-Smirnov test is that the responses of the pilots to a particular question are dis- 
tributed equally among the the response categories. A significant Kolmogorov-Smirnov result 
means that the unequal distribution of responses among the various response categories can- 
not be attributed solely to chance. The level of significance was chosen at p<.05. 

The combined responses of the familiar and unfamiliar pilots were analyzed. The question- 
naire results in Appendix A list the responses separately for the familiar and unfamiliar 
pilots. When a distinct division of opinion occurred between familiar and unfamiliar pilots, 
this is noted in the results section below. In general, the distributions of opinion for the 
familiar and unfamiliar pilots are similar. 
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4.2.1 Hardware Characteristics 


In general, the hardware characteristics for both the MFK and the DK were rated favorably 
by the pilots on the two postflight questionnaires. Most of the characteristics rated in both the 
DK and the MFK questionnaire tested significant. Eighty-eight percent of the DK responses 
and 85% of the MFK responses were good and very good. 

On the comparison questionnaire, no significant preference was noted regarding key arrange- 
ment. Half of the pilots considered the key arrangement better on the DK, and half considered 
the key arrangement on the MFK was equal to or better than the DK. All pilots felt that the 
DK was equal to or better than the MFK for key operating forces. This preference tested 
significant. 

In areas of key separation, key legend readability, key legend abbreviations, display readabil- 
ity, display brightness, and appearance of character on the display when a key is pressed, 
pilots did not indicate a significant preference for either the MFK or the DK. 

4.2.2 Logic and Task Structure 

All of the logic and task structure aspects on the DK postflight questionnaire were rated 
favorably by a significant number of pilots, with 89% of the ratings recorded as good or very 
good. On the MFK postflight questionnaire, the pilot opinion varied from very poor to very 
good; 54% of the responses overall were good or very good. Only the MFK aspects of “entering 
system commands” and “entering numeric information” showed a significant number of fa- 
vorable pilot responses. 

Pilot opinion on the comparison questionnaire was split on ease of locating the desired key, 
with no significant preference shown. Half of the pilots preferred the DK, one pilot responded 
that the MFK and DK were equal, and five pilots preferred the MFK. There was no significant 
preference shown for either the DK or the MFK in the areas of entering system commands, 
numeric information, and “knowing where you are in the logic.” 

For entering alpha information, a combination of alpha and numeric information, and in 
comparing the number of steps required, the pilots showed a significant preference for the DK 
over the MFK. Comments by several unfamiliar and one familiar pilot were made to the effect 
that alpha entry was not satisfactory on the MFK. 

4.2.3 Workload 

When asked to rate the difficulty level of operating the CDU while maintaining piloting 
functions during each task on the postflight questionnaire, a significant number of pilots 
rated the DK favorably on Tasks 1 through 4. Only Tasks 1 and 2 were rated significantly 
favorable for the MFK. When the pilots were asked to compare the ease of operation of the 
MFK and DK for each task, only the first task, that of entering the flight plan, yielded a 
significant preference for the DK. 

In terms of mental effort, four pilots responded that the DK and MFK required the same 
amount of mental effort, three responded that the DK required more, and five responded that 
the MFK required more. (All three pilots that thought the DK required more mental effort, 
were unfamiliar with FMC operation.) There was no significant preference. 
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For overall workload, 10 out of 12 pilots felt that the workload was greater on the multifunc- 
tion keyboard as compared to the dedicated keyboard. This tasted significant. 

4.2.4 Error Potential 

When the pilots rated the DK on error potential, all of the responses were borderline, good, or 
very good, and tested significant. For the MFK, 93% of the responses were borderline, good, or 
very good, and all but the “likelihood of making errors” category on the MFK questionnaire 
tested significant. 

There was no significance shown in the responses regarding error potential on the comparison 
questionnaire. Five pilots reported that there was about the same error potential on the DK 
versus the MFK. Five pilots reported that they were more likely to make errors on the MFK, 
and two reported that they were more likely to make errors on the DK. No pilot thought that 
the error potential was too high for either unit. 

4.2.5 Multifunction Keyboard CDU Design 

In the test situation, the MFK display blanks upon returning to the top level keyboard config- 
uration. Five pilots preferred this arrangement, four would rather see the previous display 
retained, and three had no preference. This did not test significant. 

Six pilots indicated that the keyboard should remain in its most recent configuration when the 
information in the scratch, pad is transferred into the display (scratch pad is emptied), one 
indicated that the keyboard should return to the next higher level, and five had other com- 
ments or no preference. 

Having dedicated keys (e.g., NEXT PAGE, PREV PAGE, CLEAR) in addition to multifunction 
keys was considered acceptable by ten pilots (two did not answer the question). This was a 
significant result. 

4.2.6 Training 

The majority of the pilots saw no problem for pilots proficient on the dedicated keyboard CDU 
to transition to the MFK, and reported no other potential training problems for the MFK. 

4.2.7 General Considerations 

All pilots considered the DK somewhat suitable or very suitable for the purpose of controlling 
the FMC, which tested significant on the 0.05 level on the DK postflight questionnaire. Al- 
though the result was not significant for the MFK, nine out of the twelve pilots considered the 
MFK somewhat suitable or very suitable for that purpose. Many comments regarding possible 
improvements for both units were made. 

As reported on the comparison questionnaire, for the tasks performed during the test, six 
pilots greatly preferred using the DK, three slightly preferred using the DK, one slightly 
preferred using the MFK, and two greatly preferred using the MFK. This did not result in 
significance at the 0.05 level. It is interesting to note that none of the pilots liked both equally. 
It is also interesting that of the pilots unfamiliar with FMC operation, four greatly preferred 
the DK, and two greatly preferred the MFK. Several pilots preferring the DK explained that 
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they disliked the number of keystrokes on the MFK, and that familiarity made it easier for 
them to operate the DK. Comments from pilots preferring the MFK cited the simpler and less 
cluttered keyboard as reasons for their choice. 

4.3 TIMELINE ANALYSIS RESULTS 

The Timeline Analysis Program (TLA) was used to examine the workload differences in using 
the MFK versus the DK during a flight. TLA is a computer model that has been used exten- 
sively to study crew workload in general. In particular, it is used to examine such workload 
aspects as ease of equipment operation, how well the operator will be able to perform the 
tasks, and task sequencing. Full description of TLA usage can be found in the TLA User’s 
Guide (ref. 7). 

The TLA comparison was done on an advanced system utilizing a dedicated keyboard CDU 
and a multifunction keyboard CDU as the interface between the pilot and a flight manage- 
ment system (FMS). The FMS was designed to represent the ’’fully integrated” next- 
generation aircraft. This FMS, therefore, had additional capabilities over current FMC 
operation, such as navigation and communication radio tuning and checklist processing. 

The mission scenario used in this analysis was developed under Phase II of the Advanced 
Flight Management CDU Study performed under NASA Contract NAS1-16300, TR B-4 (ref. 
8). Two of the three mission segments, cruise and descent, were analyzed. The location of the 
CDU was established as centered and forward of the pilot, and assumes that the copilot is 
operating the CDU with his right hand. 

Appendix D contains the graphical output from the TLA analysis. The multifunction keyboard 
CDU is designated “15-KEY CDU” and the dedicated keyboard CDU is designated “DEDI- 
CATED KYBD CDU.” The four output types are described below. 

Mission timeline: The mission timeline provides a description of each task in a procedure and 
of each event (milestone) in a flight phase. It defines the duration of each task, and presents an 
interval bar along the timeline to show when a task situation is in effect. Tasks are listed from 
top to bottom of the chart in chronological order of occurrence within the mission. Mission 
timelines for both the MFK and the DK CDUs for the cruise and descent phases are contained 
in Appendix D. 

Workload histogram: The histogram report is a plot of a channel or of the weighted average 
workload as a function of elapsed time for the entire mission. Workload histograms for each 
CDU included in Appendix D are for the right hand, internal vision, and cognitive channels, 
and for the weighted average workload. 

Channel activity: This is a bar graph that defines the average and standard deviation of the 
workload for each of nine channels, and a weighted channel average workload. The right 
hand, internal vision, and cognitive channels are of the most interest in the present analysis. 
There are two channel activity reports for each CDU type; one for the cruise segment and one 
for the approach segment. 
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Workload summary: The workload summary is a comparison between the two CDU types of 
the average workload for each channel group of the weighted average. Two summaries are 
included in Appendix D; one for each flight phase (cruise and descent). 

The mission timelines contained in Appendix D show how the multifunction and dedicated 
CDU types could be utilized in an advanced flight deck. The cruise segment consists of insert- 
ing a new 4-D flight plan to return to Atlanta via RNAV route, J-815J, STAR transition, 
MACEY 02, which was changed from SHINE 01, and insertion of Planned Time of Arrival 
(PTA) at the outer marker to runway 08, Lakeside (CATTA). The cruise segment runs from 0 to 
165 seconds. The descent segment consists of a revision of the PTA at Lakeside and a handoff 
to Atlanta approach control. This segment runs from 165 to 245 seconds. As can be seen in the 
timelines, neither the cruise nor descent phase requires the entry of alpha characters. 

The workload histograms for the multifunction and dedicated keyboard CDUs show similar 
patterns of workload on the internal vision, right hand, and cognitive channels. The weighted 
channel average histograms show both CDUs well below the 80% workload limit. The multi- 
function CDU weighted channel average histogram shows that the workload is more evenly 
distributed over the mission as compared to the dedicated CDU histogram. The MFK does not 
exceed 40% workload at any given point, while the DK peaks at over 70% workload. 

The channel activity summaries for the multifunction and the dedicated CDUs show very 
similar percent workload values for the different workload channels. Similarly, the workload 
summaries, which compare the totals of the workload groups for the DK and the MFK, reveal 
little difference between the two CDU types. In the cruise segment, the total vision for the 
MFK is slightly higher than that for the DK, but this difference is cancelled by the slightly 
lower value on the MFK as compared to the DK for total communications, resulting in approx- 
imately the same weighted average. In the descent phase, the workload summary shows very 
little difference between the MFK and DK for the workload groups and for the weighted 
average. 
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5.0 DISCUSSION AND CONCLUSIONS 


For the operations performed in this test, the multifunction keyboard CDU required more 
keystrokes, in general, than did the dedicated keyboard CDU. This was a direct result of the 
double keystroke required to input alpha information. The pilots performed tasks faster on the 
DK than on the MFK. There was no significant interaction between pilot familiarity and CDU 
type, however, which indicates that even for pilots new to both units and unfamiliar with the 
overall FMC operation, there is no advantage gained by using the MFK for this particular 
operation. 

In evaluating the differences between the multifunction and dedicated keyboards for tasks in 
general, the average time per keystroke was analyzed. There was no significant difference 
between the DK and the MFK for pilots, in general. Pilots who were unfamiliar with the FMC 
operation, however, had a lower average keystroke time on the MFK than on the DK; the 
opposite was true for the familiar pilots. This result, however, is confounded by the fact that 
whereas the pilots who were not familiar with FMC operation were equally familiar with the 
two CDU types, the pilots familiar with the FMC operation were much more familiar with the 
DK than with the MFK. Several familiar pilots commented that they knew the keyboard on 
the DK and were more comfortable with it. It is possible that after experience with the MFK 
to equal their experience with the DK, these pilots would show the same tendency as the 
others (i.e., they would have a lower average keying time on the MFK than on the DK). 

There was no significant difference in terms of average keying times due to the type of CDU 
(MFK versus DK). The interaction of pilot familiarity with CDU type may have masked any 
difference in average keying times between the two CDU types, however. If in fact all pilots 
were equally familiar with both units, there may have been a significantly lower average 
keystroke time on the MFK as opposed to the DK. 

Familiarity with the conventional control is an important consideration. Pilots who were 
familiar with the DK tended to prefer it to the MFK. Some commented that they liked it 
better because they were used to it and could anticipate their next action as they were com- 
pleting the current action. Although the keyboard configuration on the MFK is changing 
constantly, the same keys pop up in the same location, thus enabling a pilot to “memorize” 
key location as they have done with the DK. If they were able to use the MFK until they 
attained the same degree of familiarity as on the DK, more pilots may have preferred the 
MFK. 

The demands of flight activity had a significant effect on the pilot performance on both CDUs. 
Average keying time when hand flying during turbulence was almost twice that which oc- 
curred on the ground. Performance on the MFK and the DK CDU seem to be affected equally 
by flight activity, as there was no significant interaction between flight activity and CDU 
type. The relatively higher average keying times for the unfamiliar pilots during the high 
flight activity conditions is not easily explained. It may be that the high level of activity 
placed greater stress on those pilots who were not extremely familiar with FMC operation. 

Errors could not be analyzed properly due to the fact that the MFK CDU was not programmed 
to provide error correction. There was however, some indication that there was no significant 
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difference between the two units in terms of error probability, as the number of error se- 
quences per keystroke on the MFK was not significantly different from the number of error 
sequences per keystroke on the DK. 

No differences were found between the two units when the pilot flight performance (as mea- 
sured by the RMS error of the flight parameters) was analyzed. These parameters may not be 
sensitive to small differences in performance due to the factors of interest. A recent study 
showed that primary task measurements, such as control movement and the mean square of 
pitch and roll, did not exhibit reliable changes as a function of mediational (e.g., cognitive) 
load (ref. 9). 

It may also be that the pilots were concentrating mostly on operating the CDU as opposed to 
flying. One recent study compared data manually entered (via multifunction control and dedi- 
cated keyboard) to vocally entered. “In the situation when data entry was done manually the 
pilots would concentrate most of their information processing on the data entry task and 
flying performance would suffer; however, when data entry was done vocally the pilots could 
still concentrate on the flying task and enter data while maintaining good flight control” (ref. 
10). Several pilots in the present study commented that the experimental design was not 
realistic in the sense that data would “never” be entered manually by the pilot flying. This 
however, is a standard technique in measuring increases in workload, i.e., the performance on 
a secondary task (flying) is evaluated with respect to changes in a primary task (FMC input). 
Also, the flying qualities of the simulator may have aided in masking any differences between 
the two units. 

For the examination of input type, Task 6 was chosen for analysis. Pilot familiarity and key 
type were found to be significant. Numeric input time was less than alpha input time, which 
in turn was less than system command or line select input times. Interestingly, CDU type was 
not found to be significant. It was expected that there would be a significant interaction 
between the CDU type and key type, showing a relative difference between the MFK and the 
DK in terms of input type (in particular, input of alpha information). Although the F value of 
this interaction is fairly high, it does not meet the criteria level of p<.05. It may be that 
because the task chosen for analysis was the high flight activity, high task complexity condi- 
tion, this interaction was masked. 

Pilot opinion reported through the questionnaires revealed some problems with the MFK. 
Hardware characteristics were considered good for both units, with the exception of the MFK 
key operating forces. The MFK switches hinged at the top and worked best if pressed near the 
bottom of the switch. Pressing too close to the hinge required excessive force. The logic and 
task structure on the MFK were not rated significantly favorable in general. Only the catego- 
ries of entering system commands and numeric information were rated favorably. A signifi- 
cant number of pilots preferred the DK to the MFK in terms of the number of steps required, 
and for entering alpha information or a combination of alpha and numeric information. Also, 
overall workload was perceived as higher on the MFK than on the DK. 

The results of the timeline analysis shows that the MFK and the DK, when used in an ad- 
vanced flight deck concept, impose similar workload levels. The advanced flight deck concept 
did not contain any alpha input. The workload was more evenly distributed over the mission 
for the MFK than for the DK, although it may be possible to shift the timing of operations on 
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the DK to smooth out the workload peaks. Thus, in terms of workload on the advanced flight 
deck, the MFK performs as well as or better than the DK. 

The results of this test indicate that not all tasks performed on a dedicated unit are amenable 
to direct translation to a multifunction unit. Although the task used for comparison in this 
test may not be a suitable candidate for a multifunction keyboard, several possible means of 
improving the function and pilot acceptance of multifunction keyboards are evident. The num- 
ber of choices of top level and in subsequent configurations could possibly be reduced depend- 
ing on the phase of flight. This has been recommended by previous studies as a design 
guideline and was discussed in Section 3.3. For the FMC tasks used as a comparison in this 
test, a reduction depending on flight phase would be very limited. In operations requiring 
numerous levels of keyboard paging, reducing the number of choices could make a significant 
improvement. 

It is possible to use the line select keys adjacent to sections of the display area to call up the 
keyboard appropriate for that selection. This method was used in Phase II of the Advanced 
Flight Management CDU Study. For tasks such as the FMC operation, where the keyboard 
was programmed to bring up the alpha configuration automatically when appropriate, this 
would actually add keystrokes to the scenarios. For other applications requiring the use of a 
variety of entry keyboard configurations, however, this method could be easier than paging 
through to the desired configuration. When asked about this method, overall pilot opinion was 
neither significantly favorable nor significantly unfavorable. 

It is also possible to mix dedicated keys with multifunction keys; the appropriate number of 
each type is dependent on the task. There were three keys in FMC operation that would have 
generally appeared on the multifunction keyboard (e.g., CLEAR, NEXT PAGE, and PREV 
PAGE). It was felt appropriate to place keys of this nature on dedicated keys. Although they 
were not functional during this test, this arrangement served to free up keys on the multifunc- 
tion keyboard. A significant number of pilots responded favorably when asked about this. 

An increasing use of artificial intelligence would increase the desirability of multifunction 
keyboards over dedicated keyboards. A multifunction keyboard would allow a “smart system” 
to be used to its best advantage, i.e., to reduce the amount of irrelevant data confronting the 
pilot. 

Functions chosen for implementation on a multifunction CDU should be selected on the basis 
of how the tasks will be structured as compared to the traditional dedicated controls. As in the 
case of the FMC operation, tasks that require significantly more keystrokes to accomplish on 
the MFK should be avoided. Inclusion of a large amount of alpha entry increases the number 
of keystrokes to a level where performance times are significantly higher on the MFK than on 
the DK. Average time per keystroke was not significantly different between units, and pilots 
unfamiliar with FMC operation had a lower average keystroke on the MFK. Thus, if a func- 
tion can be accomplished without a substantial increase in the number of keystrokes on an 
MFK as compared to a DK, it is a suitable candidate for MFK implementation. 
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APPENDIX A 

QUESTIONNAIRE SUMMARY 
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Appendix A contains the summary of the responses and comments of all 12 pilots. Re- 
sponses for familiar and unfamiliar pilots are listed separately. For a given response category, 
the total number of familiar pilots with that response is listed first and the total number of 
unfamiliar pilots is listed second, separated by a slash. The Kolgomorov-Smirnov D statistic is 
listed in a column on the right-hand side of the page. Written comments are listed by pilot 
number below each question. Comments written by familiar and unfamiliar pilots are desig- 
nated with an F and a U, respectively, after the pilot number. 
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MULTIFUNCTION KEYBOARD CDU— POSTFLIGHT QUESTIONNAIRE 


INSTRUCTIONS 

The following questions pertain to the multifunction keyboard CDU. Please mark your 
answers to the questions as indicated. Space is provided at the bottom of each page for com- 
ments. Any comments or suggestions you would like to make would be appreciated. Also, use 
the comments space to enumerate any operational difficulties encountered during the test. 
Please be as specific as possible. If you have any questions, ask one of the test conductors. 
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HARDWARE CHARACTERISTICS 


For questions 1 through 6, mark your answers by putting X in the appropriate column. 
1. Rate the following factors for the multifunction keyboard CDU. 



Very 

Poor 

Poor 

Border- 

line 

Good 

Very 

Good 

D 

Keyboard : 







Key arrangement 

0/1 

1/0 

1/2 

3/1 

1/2 

.23 

Key operating forces 

1/0 

1/0 

2/2 

1/3 

1/1 

.23 

Key separation 



1/0 

2/3 

3/3 

.52* 

Keyboard location 

1/0 


1/1 

2/4 

2/1 

.35 

Key legend readability 



0/1 


6/5 

.72* 

Key legend abbreviations 



0/1 

3/3 

3/2 

.52* 

Display : 







Display readability 




2/5 

4/1 

.60* 

Display format 



0/1 

6/4 

0/1 

.52* 

Display information 
density 




b/b 

1/1 

.6U* 

Display brightness+ 




5/3 

1/2 

.60* 

Control /display features: 







Relationship of controls 
to dfisplay 


0/1 

0/1 

6/3 

0/1 

.43* 

CDU location 

1/1 


1/0 

3/5 

1/0 

.35 

Appearance of character 
on the display when a key 
is pressed 




4/4 

2/2 

.bU* 


+ No answer from one pilot 
* p < .05 
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COMMENT: 


Subject 4 (U): Display format— It seems to me that there could be better differentiation be- 
tween heading labels on the display and selected data. Keyboard display— would be better on 
middle console, as it is on aircraft. 

Subject 8 (U): Relationship of controls to display— parallax problem. 

Subject 10 (F): With experience my acceptability of this keyboard would improve dramatically. 
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LOGIC AND TASK STRUCTURE 


2. Rate the following information entry factors for the multifunction keyboard CDU. 



Very 

Poor 

Poor 

Border- 

line 

Good 

Very 

Good D 

Entering information: 






Locating the desired key 


1/0 

2/3 

3/2 

0/1 .32 

Entering system commands 


1/0 

0/1 

5/3 

0/2 .52* 

Entering alpha 
information 

1/2 

1/1 

1/1 

2/0 

1/2 .05 

Entering numeric 
i nformation 


1/0 

0/1 

3/3 

2/2 .52* 

Entering a combination 
of alpha and numeric 

1/2 

3/0 

1/2 

1/1 

0/1 .15 

3. Rate the overall task structure for the multifunction keyboard. 



Logical steps 


1/0 

1/2 

3/2 

1/2 .32 

Task complexity 

2/0 

0/1 

2/1 

1/2 

1/2 .15 

Number of steps required 

2/1 

0/1 

0/1 

4/4 

.20 

“Knowing where you are" 
in the logic+ 


1/0 

4/3 

0/2 

1/0 .30 


+ No answer from one pilot 
*p < .05 

COMMENT: 


Subject 4 (U): The number of steps required to enter alpha data is really unacceptable. There 
should be at least 26 keys. This concept has a disadvantage in that switches change function, 
so you can’t “memorize” the keyboard. But the advantage is that only a few key possibilities 
are presented at a time, and they are large and clearly labeled. 

Subject 6 (U): With a little practice I felt (surprisingly rapidly so, to me, in view of my limited 
previous exposure to the multifunction keyboard) basically comfortable operating the system. 

Subject 7 (F): In time would recall the use of L and R keys after a numeric entry (i.e., RW13R). 

Subject 8 (U): Locating the desired key would rapidly increase with use. Double entry of alpha 
is not an easy task while flying. 

Subject 9 (F): Should have “A” and “B” on numeric page for above and below altitudes. 


40 



Subject 11 (U): Entering alpha information required two steps. Also required two different 
keys most of the time. This increased chance for errors, takes more time. Entering alpha 
numerics even more cumbersome because you had to switch down from the alpha page to 
numeric page. 
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WORKLOAD 


4. Rate the difficulty level of operating the multifunction keyboard CDU while maintaining 
piloting functions during the following test phases. 



Very 


Border- 


Very 


Poor 

Poor 

line 

Good 

Good D 

Prefl ight : 






Enter flight plan 


1/0 

1/0 

3/3 

1/3 .52* 

Performance 

initialization 



2/U 

2/3 

2/3 .43* 

Cruise: 






Modify route 

1/0 

0/1 

1/1 

3/3 

1/1 .27 

Enter altitude 
restriction 

2/0 


1/1 

2/4 

1/1 .27 

Approach : 






Set up approach 

1/0 

0/1 

2/0 

2/4 

1/1 .27 

Time/distance to 
crossing radial 

1/0 

0/1 

1/1 

3/3 

1/1 .27 

5. Rate the acceptability of workload factors for the multifunction keyboard. 


Mental effort required 

1/0 

1/1 

1/0 

2/4 

1/1 .27 

Overall workload 

1/0 

1/2 

1/0 

2/4 

1/0 .27 

* p < .05 






COMMENT: 







Subject 4 (U): Requiring two or three switch actuations for each letter input is a serious pilot 
distraction. 

Subject 6 (U): I mark very good and good in view of the rapidity with which one gains comfort 
with the system. 

Subject 8 (U): Without an instructor alongside there would be a lot of confusion with the little 
knowledge I have of the system. 
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ERROR 


6. Rate the multifunction keyboard on error potential. 



Very 

Poor 

Poor 

Border- 

line 

Good 

Very 

Good 

D 

Likelihood of 
errors+ 

maki ng 

2/0 

4/1 

U/4 


22 

Ease of error 

detection+ 


2/3 

3/1 

1/1 

.40* 

Ease of error 

correction++ 


2/2 

2/0 

1/3 

.40* 


+ No answer from one pilot 
++No answer from two pilots 
* p < .05 

COMMENTS: 

Subject 4 (U): The extra switch activations required detracts from flying the aircraft, and 
leaves less time to double-check. 

Subject 8 (U): Unable to evaluate as the test would take no errors would definitely like some 
sort of check method to verify that entered data is correct. The initial flight plan would not be 
a problem but making changes on Arr/Dep could get a bit confusing and subject to wrong 
route flying or being directed toward wrong Route/Act restrictions. 


Subject 10 (F): Did not try error correction. 
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GENERAL CONSIDERATIONS 


7. How familiar were you with multifunction keyboard CDUs prior to participating in this 
study? 

Very familiar 

1/0 Somewhat familiar 
5/6 Unfamiliar 

8. How suitable is the multifunction keyboard CDU for the purpose you saw demonstrated 
today? 

2/2 Very suitable 

2/3 Somewhat suitable 

1/0 Borderline 

1/0 Somewhat unsuitable 

0/1 Very unsuitable 

9. What improvements should be made to the multifunction keyboard CDU? 

Subject 2 (F): Less pressure on keys. Make a next page selection available for alpha instead of 
three per key. Have keys in preflight, in flight, and approach sequence. 

Subject 4 (U): Have 12 to 15 large keys, and enough small ones to fill out an entire alphabet. 

Subject 5 (F): Another alphanumeric selection scheme 

Lower touch pressure 

More and smaller keys are possible 

Some dedicated high use keys 

Subject 6 (U): Perhaps a little less pressure required to activate. I see this in relation to 
turbulence and chop conditions, not in the on-the-ground regime. 

Subject 7 (F): Softer touch on keyboard 
Larger letters to numbers in keys 

More than one way to switch from one function to another without “being wrong.” 

Subject 8(U):. Alpha selection— double key push especially while flying is unsatisfactory. +/- 
key confusing but would clear up with use. 

Subject 10 (F): Have the page blink when you make an entry. 

Subject 11 (U): Simplify alpha keyboard to one step— one letter capability. 

Please use the space below to write any additional comments you might have for the multi- 
function keyboard CDU. 

Subject 1 (F): Unnecessary pilot workload— a simple alpha and numeric entry requires several 
inputs to the CDU as opposed to DAKs. 
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Subject 3 (U): The line select keys were hard for me to see which line they lined up with. 
Subject 7 (F): I think I could learn to like it. 

Subject 9 (D): In general, very good and easy to learn to use. 

Subject 11 (U): The keyboard seems quite large. This makes it easy to read and operate, but 
cockpit panel space is really limited in commercial airplanes. 
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DEDICATED KEYBOARD— CDU POSTFLIGHT QUESTIONNAIRE 


INSTRUCTIONS 

The following questions pertain to the dedicated keyboard CDU. Please mark your answers to 
the questions as indicated. Space is provided at the bottom of each page for comments. Any 
comments or suggestions you would like to make would be appreciated. Also, use the com- 
ments space to enumerate any operational difficulties encountered during the test. Please be 
as specific as possible. If you have any questions, ask one of the test conductors. 
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HARDWARE CHARACTERISTICS 


For questions 1 through 6, mark your answers by putting an X in the appropriate column. 
1. Rate the following factors for the dedicated keyboard CDU. 



Very 

Poor 

Poor 

Border- 

line 

Good 

Very 

Good 

D 

Keyboard : 







Key arrangement 



0/1 

b/4 

1/1 

.52* 

Key operating forces 




4/2 

2/4 

.60* 

Key separation 



1/1 

S/3 

0/2 

.43* 

Keyboard location 



2/0 

4/6 


.43* 

Key legend readability 



2/2 

4/4 


.40* 

Key legend 
abbreviations 




6/5 

0/1 

.6U* 

Display : 







Display readability 



1/1 

2/5 

3/0 

.43* 

Display format 



0/1 

6/4 

0/1 

.52* 

Display information 
density 



0/1 

5/4 

1/1 

.52* 

Display brightness 




5/4 

1/1 

.60* 

Control /displ ay features: 







Relationship of 
controls to display 



0/1 

6/5 


.52* 

CDU location 

0/1 

1/0 

1/1 

4/4 


.27 

Appearance of character 
on the display when a 
key is pressed 




6/5 

0/2 

.60* 

* p < .05 
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COMMENT: 


Subject 3 (U): It would help to have the CDU on the pedestal so the flight instruments would 
be in your scan. 

Subject 4 (U): Display readability would be better if labels and data were better differentiated. 
Key legend readability function keys are too small for labels. 

Subject 8 (U): First looking at the dedicated CDU screen did not like the char. pres, but after 
using found it OK. Changed size of altered data very desirable. 
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LOGIC AND TASK STRUCTURE 


2. Rate the following information entry factors for the dedicated keyboard CDU. 

Very Border- Very 

Poor Poor line Good Good D 


Entering information: 
Locating the 


desired key 


U/l 

6/4 

U/l 

.52* 

Entering system 
commands 



6/4 

U/2 

• 6U* 

Entering alpha 
information 


2/1 

4/2 

0/3 

.40* 

Entering numeric 
i nformati on 


1/0 

4/3 

1/3 

.52* 

Entering a combination 
of alpha and numeric 


2/1 

2/2 

2/3 

.40* 

Rate the overall task structure for the dedicated keyboard. 




Logical steps 


0/2 

4/4 

2/U 

.43* 

Task complexity 


0/1 

5/4 

1/1 

.52* 

Number of steps 
requi red 



5/b 

1/1 

.60* 

"Knowing where you 
are" in the logic 

0/1 


b/b 

1/0 

.52* 


* p < .05 
COMMENT: 

Subject 4 (U): It would help in your handout to know where one is flying (route, etc.) to make 
the entries more logical. 

Subject 4 (U): The dedicated keyboard seemed to be a little more complex in operation or 
functions. However, I believe that this would improve rapidly with practice. 

Subject 8 (U): Knowing where you are in the logic is a little tough doing just bits and pieces. 
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WORKLOAD 


4. Rate the difficult level of operating the dedicated keyboard CDU while maintaining pilot- 
ing functions during the following test phases. 



Very 


Border- 


Very 



Poor 

Poor 

line 

Good 

Good 

D 

Prefl ight : 







Enter flight plan 




3/2 

3/4 

• bU* 

Performance 
initial i zation 




3/2 

3/4 

.60* 

Cruise : 







Modify route 


0/1 

1/0 

4/3 

1/3 

.43* 

Enter altitude 
restriction 


0/1 


5/4 

1/1 

.52* 

Approach : 







Set up approach 


u/l 

1/1 

5/3 

0/1 

.35 

Time/distance to 
crossing radial+ 


0/1 

1/1 

3/3 

1/1 

.33 

5. Rate the acceptability of workload factors for the dedicated keyboard. 



Mental effort required 



0/2 

5/4 

1/0 

.43* 

Overall workload 



0/2 

6/4 


.43* 

+ No answer from 1 pilot 
* p < .05 







COMMENT: 








Subject 3 (U): The workload would be reduced if one was more familiar with not only the 
dedicated keyboard but also the routing one is to fly. 

Subject 8 (U): Found this keyboard much preferred to the multifunction for data entry. 
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ERROR 


6. Rate the dedicated keyboard on error potential. 




Very 

Poor 

Poor 

Border- 

line 

Good 

Very 

Good 

D 

Likelihood of 
errors 

making 



2/1 

4/5 


. 4(J* 

Ease of error 

detection 



1/3 

5/1 

0/2 

.40* 

Ease of error 

correction 




6/4 

0/2 

• 6U* 


* p < .05 

COMMENT: 

Subject 3 (U): I feel the errors encountered are due largely to being unfamiliar with the unit. 

Subject 10 (F): Error likelihood high when you first start, less when you gain experience with 
the system. 
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GENERAL CONSIDERATIONS 


7. How familiar were you with dedicated keyboard CDUs prior to participating in this 
study? 

5/0 Very familiar 
1/0 Somewhat familiar 
0/6 Unfamiliar 

8. How suitable is the dedicated keyboard CDU for the purpose you saw demonstrated 
today? 

6/4 Very suitable 
0/2 Somewhat suitable 

Borderline 

Somewhat unsuitable 

Very unsuitable 

*p < .05 

9. What improvements should be made to the dedicated keyboard CDU? 

Subject 4 (U): Make function keys selectable to illustrate suitable functions for the task at 
hand. Also, make them larger with better labels. 

Subject 5 (F): More prominent lettering on the keys (i.e., brighter paint, larger letters, etc.). 

Subject 6 (U): I found the alpha key locating to be a little difficult in comparison to the 
multifunction keyboard. 

Subject 7 (F): Highlight page on CRT. 

Subject 10 (F): Progress page is too cluttered. 

Please use the space below to write any additional comments you might have for the dedicated 
keyboard CDU. 

Subject 1 (F): Simple direct entry keys are fast and simple in operation. Less workload than 
multientry keys. 

Subject 6 (U): I found the key pressure required to be ideal as contrasted to the multifunction. 

Subject 8 (U): Am probably biased against excessive automation of flying functions. Removing 
too many tasks reduces the pilots to computer operators except when some part is inoperable 
and then it is back to manual. 

Subject 12 (U): With experience, this operation should be easy. I like it. 
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FLIGHT MANAGEMENT COMPUTER CDU PROGRAM QUESTIONNAIRE 


INSTRUCTIONS 

The following questions pertain to both the multifunction keyboard CDU and the dedicated 
keyboard CDU. Please mark your answers to the questions as indicated. Space is provided at 
the bottom of each page for comments. Any comments or suggestions you would like to make 
would be appreciated. Also, use the comments space to enumerate any operational difficulties 
encountered during the test. Please be as specific as possible. If you have any questions, ask 
one of the test conductors. 
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HARDWARE CHARACTERISTICS 


For questions 1 through 4, place an X in the column which best describes the statements 
comparing the dedicated keyboard CDU (DK) to the multifunction keyboard CDU (MFK). 

1. How do the dedicated keyboard CDU and the multifunction keyboard CDU compare in 
terms of the following hard characteristics? 



DK 

DK 

DK & 

MFK 

MFK 



Much 

Slightly 

MFK 

Slightly 

Much 



Better 

Better 

Equal 

Better 

Better 

D 

Keyboard : 







Keyboard arrangement 

2/2 

1/1 

0/1 

3/0 

0/2 

.13 

Key operating forces 

3/3 

3/1 

0/2 



.43* 

Key separation 


0/1 

2/1 

1/2 

3/2 

.32 

Key legend readability 


0/1 

0/2 

5/2 

1/1 

.3b 

Key legend abbreviations 

0/1 


2/1 

3/2 

1/2 

.20 

Display : 







Display readability 


2/1 

3/3 

1/2 


.20 

Display brightness 


0/1 

6/3 

0/2 


.32 

Control/display features: 







Appearance of character 
on the display when a key 
is pressed 


1/1 

b/4 

0/1 


.31 

* p < .05 








COMMENT: 

Subject 4 (U): Both displays need more differentiation between headings and data. 

Subject 8 (U): I did not like the touch of the MFK as it seemed to take too much pressure to 
activate the key, especially if the key was pressed too close to its hinge. I could also be biased 
against the MFK as I was involved with it during the other test (Flight Status Monitor) and 
see an attempt to have one keyboard serve two very different functions, which could be a real 
operational problem if multifunction problems were being attended to as well as a navigation 
problem. 

Subject 11 (U): DK key arrangement better than the MFK because the MFK alpha keyboard 
requires too many steps. 
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LOGIC AND TASK STRUCTURE 


2. How do the dedicated keyboard CDU and the multifunction keyboard CDU compare 

for the following information entry factors? 



DK 

Much 

Better 

DK 

Slightly 

Better 

DK & 
MFK 
Equal 

MFK 

Slightly 

Better 

MFK 

Much 

Better 

D 

Locating the desired key 

2/3 

1/0 

1/0 

2/1 

0/2 

.21 

Entering system commands 

2/1 

1/2 

3/1 

0/2 


.23 

Entering alpha information 

3/4 

3/0 


0/1 

0/1 

.43* 

Entering numeric information 

2/2 

2/1 

1/1 

1/2 


.20 

Entering a combination of 
alpha and numeric 

3/4 

3/0 


0/1 

0/1 

.43* 

3. How did the overall task structure compare for the 
multifunction keyboard CDU? 

dedicated keyboard CDU versus the 

Logical steps 

2/0 

1/1 

3/3 

0/1 

0/1 

.23 

Task complexity 

2/0 

1/2 

2/3 

1/0 

0/1 

.23 

Number of steps required 

3/3 

2/2 

1/0 

0/1 


.43* 

"Knowing where you are" 
in the logic 

1/1 

1/1 

4/3 

0/1 


.23 


* p < .05 
COMMENT: 

Subject 8(U): The alpha data entry for the MFK is unsatisfactory due to requiring a double 
entry for each character. 

Subject 11 (U): Alpha and numeric keyboards were on different pages on the multifunction. 
This was awkward to enter 16L. 
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WORKLOAD 


4. Compare the ease of operating the dedicated keyboard CDU and the multifunction key- 


board CDU while maintaining piloting functions during the following test phases: 

DK DK DK & MFK MFK 

Much Slightly MFK Slightly Much 

Better Better Equal Better Better 

D 

Prefl ight : 

Enter flight plan 

2/0 

2/3 

2/3 



.4U* 

Performance initialization 
Cruise : 

2/0 

1/3 

3/2 

0/1 


.31 

Modify route 
Entering altitude 

2/1 

3/1 

1/3 


0/1 

.31 

restri ction 
Approach : 

2/0 

3/2 

1/3 


0/1 

.31 

Set up approach 
Time/distance to 

2/2 

1/1 

3/1 

0/2 


.23 

crossing radial 
* p < .05 

COMMENT: 

2/1 

1/2 

3/1 

0/2 


.23 


Subject 3 (U): The MFK with the larger keys and printing seem to make it easier to operate in 

flight. It takes less attention and concentration to operate. 

Subject 4(U): The added keystrokes needed to utilize the MFK, while hand flying the aircraft 

made the operation unsafe. 

For questions 5 through 13, place an X before the statement that best describes your answer 

to the question. 

5. Which unit required greater mental effort during operation? 

0/1 Dedicated keyboard CDU required much more mental effort than the multifunction key- 
board CDU 

0/2 Dedicated keyboard CDU required slightly more mental effort than the multifunction 
keyboard CDU 

3/1 Both required about the same amount of mental effort 

2/0 Multifunction keyboard CDU required slightly more mental effort than the dedicated 
keyboard CDU 
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1/2 Multifunction keyboard CDU required much more mental effort than the dedicated key- 
board CDU 

Was the mental effort required excessive for either unit?+ 

0/2 Yes 
6/3 No 

+ No answer from one pilot 

COMMENT: 

Subject 3 (U): I felt the mental effort with the DK was too distracting while one is flying. 

Subject 4 (U): It became a dangerous distraction while hand flying, particularly the MFK. 

Subject 10 (F): No, but with the MFK it might be harder to come back to after a long distrac- 
tion. 

6. Which of the following statements on the overall workload applies: 

Workload on the dedicated keyboard CDU was much greater as compared to the multi- 
function keyboard CDU. 

0/1 Workload on the dedicated keyboard CDU was slightly greater as compared to the multi- 
function keyboard CDU. 

0/1 Workload was about the same for the dedicated keyboard CDU and the multifunction 
keyboard CDU. 

4/3 Workload on the multifunction keyboard CDU was slightly greater as compared to the 
dedicated keyboard CDU. 

2/1 Workload on the multifunction keyboard CDU was much greater as compared to the 
dedicated keyboard CDU. 

Was the overall workload required excessive for either unit?+ 

2/2 Yes 
3/3 No 
1/0 Other: 

Subject 3 (U): Both require distraction from flying when some of the operations (such as alti- 
tude and speed restrictions) could be accomplished with the autopilot. 

+ No answer from one pilot 
*p < .05 

COMMENT: 

Subject 1 (F): Multifunction for some entries requires three separate key functions. Example- 
entering RW13R. 
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Subject 2 (F): MFK would be for some operators. 

Subject 4 (U): Entering alpha information on the MFK was unreasonably complicated. 

Subject 5 (U): Combination alpha/numeric on multifunction keyboard took longer. 

Subject 8 (U): It is possible that changes in approaches, etc., made in the terminal phase of a 
flight would not be entered into the system. While flying the DK I know where I am, as 
compared to entering a last minute change and then being concerned if the “machine” is 
giving me proper guidance. An enroute change in altitude would be no problem. It is the 
changes in departure or arrival routings made in congested areas that cause me concern as far 
as having to enter the information. 
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ERROR POTENTIAL 


7. Which of the following statements on error potential applies: 

Much more likely to make errors on the dedicated keyboard CDU than on the multifunc- 
tion keyboard CDU 

0/2 Somewhat more likely to make errors on the dedicated keyboard CDU than on the multi- 
function keyboard CDU 

4/1 About the same error potential for the dedicated keyboard CDU and the multifunction 
keyboard CDU. 

2/2 Slightly more likely to make errors on multifunction keyboard CDU than on the dedi- 
cated keyboard CDU 

0/1 Much more likely to make errors on multifunction keyboard CDU than on the dedicated 
keyboard CDU 

Do you think the error potential was too high for either unit?+ 

Yes 

6/4 No 
0/1 Other: 

Subject 3 (U): Not being familiar with the use of either unit, I cannot make an accurate 
comment. 

+ No answer from one pilot 
*p < .05 

COMMENT: 

Subject 4 (U): As a first -time user, I made only a couple of errors. On the DK, the errors tended 
to be in selecting modes. The MFK helps avoid this, but the 2-stroke alpha inputs are confus- 
ing and error producing, so I must rate this a draw. 

8. Although the multifunction keyboard CDU was not programmed to provide realistic 
error scenarios, which of the following statements applies concerning error detection 
during this test: 

1/0 Much easier to detect errors on dedicated keyboard CDU 
0/2 Slightly easier to detect errors on dedicated keyboard CDU 

4/3 About the same ease of error detection for the dedicated keyboard CDU and the multi- 
function CDU 

1/0 Slight easier to detect errors on the multifunction keyboard CDU 
0/1 Much easier to detect errors on the multifunction keyboard CDU 

COMMENT: 

Subject 5 (F): Line select keys on multifunction CDU were not aligned visually with the 
display lines and probably contributed to my error(s) in line select entry. Multifunction keys 
required too high a force even using the lower edge. 
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9. Which of the following statements on error correction applies +: 

2/1 Much easier to correct errors on dedicated keyboard CDU 
0/1 Slightly easier to correct errors on dedicated keybard CDU 

2/2 About the same ease of error correction for the dedicated keyboard CDU and the multi- 
function keyboard CDU 

1/2 Slightly easier to correct errors on the multifunction keyboard CDU 
Much easier to correct errors on the multifunction keyboard CDU 

+ No answer from one pilot 

COMMENT: 

Subject 7 (F): Clear key was not functional on the MFK. 

Subject 8 (U): I don’t like the MFK because of the double key press requirement. 

Subject 10 (F): Didn’t correct any errors on the MFK, therefore no error correction comparison 
possible. 
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MULTIFUNCTION KEYBOARD CDU DESIGN 


The design of the multifunction keyboard CDU in this test reflects the current operation of the 
737-300 CDU. There are many changes that could be made to the operation of the multifunc- 
tion keyboard. The questions below are being asked to obtain your insights and preferences 
regarding multifunction keyboard operation. 

10. In the design for this test, the display blanks upon returning to the top level keyboard 
configuration. Did you find this helpful or would you prefer the previous information to 
remain in the display? 

2/3 Should blank upon returning to top level 
3/1 Should retain previous display 
0/2 No preference 
1/0 Other (specify); 

Subject 5 (F): Would probably have a preference with more experience of unit in the field. 

11. What should happen to the keyboard configuration when information entered into the 
scratch pad is transferred into the display (i.e., the scratch pad is emptied)? 

3/3 The keyboard should remain in its most recent configuration (i.e., alpha or numeric) 

0/1 The keyboard should return to the next higher level 
0/2 No preference 
3/0 Other (specify): 

Subject 5 (F): Need more experience. 

Subject 7 (F): The keyboard should return to the top level. 

Subject 10 (F): The answer is different for different phases of flight. In some cases (PERF), stay 
in recent configuration; in others maybe go to next higher level. 

COMMENT: 

Subject 4 (U): I would much prefer a 26 through 28 key keyboard with both alpha and numer- 
als on the key faces, with a shift button to select one or the other. This display should remain 
when the scratch pad is emptied. 

Subject 5 (F): More artificial intelligence is probably possible than was used in the demonstra- 
tion. This might improve the multifunction ratings. 

12. One possible method of reconfiguring the keyboard for data entry to the scratch pad is to 
use the line select key next to the data field to be filled. Pressing the line select key 
would change the keyboard to the appropriate configuration (i.e., alpha, numeric, or a 
choice if data can be entered both ways for that particular field). Would this design be: 

2/0 Very satisfactory 
1/4 Satisfactory 
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1/1 Borderline 
2/0 Unsatisfactory 
Very unsatisfactory 

Subject 1 (F): Increased workload. 

13. The functions of NEXT PAGE, PREV PAGE, and CLEAR were located on dedicated keys 
beneath the display area, rather than on multifunction keys. Did you find this arrange- 
ment to be: 

2/4 Very satisfactory 
2/2 Satisfactory 

Borderline 

Unsatisfactory 

Very unsatisfactory 

+ No answer from two pilots 
*p < .05 

COMMENT: 

Subject 6 (U): While we did not use these keys to any extent in this program, I can see their 
“dedicated” status to be appropriate especially in an online, inflight, actual situation. 

14. The prompt area and its associated line select key on the dedicated keyboard CDU is used 
as a “multifunction” area. For example, the ACTIVATE > prompt appears on the Route 
page at the bottom of the display; pressing the line select key next to it tells the FMC to 
activate the data. Can you think of any functions that should be treated in this manner 
on the multifunction keyboard CDU display? 

Subject 2 (F): Use same idea to prompt sequence (i.e., Pos Init, Rte, Perf, Dpt). 

Subject 4 (U): Yes, similar function. It is useful as a signal to the pilot that the needed input is 
completed. 

Subject 5 (F): Why not instead eliminate the ACTIVATE and use the execute key? Why do we 
need to do both? 

Subject 10 (F): There might be some climb modes or descent modes that would be better off as 
a line select key. 

COMMENT: 

Subject 5 (F): On both displays, it is difficult to immediately find the alpha characters. Per- 
haps a different alpha layout would help (e.g., two long rows of keys A-M, M-Z, or vowels 
colored, or alpha keys bordering the CDU keyboard, etc.). The numeric keys were in a familiar 
layout and were easy to find for both. The multifunction numeric was extremely easy to use. 
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TRAINING 


15. Do you think there would be any problem for pilots proficient on the dedicated keyboard 
CDU to transition to a multifunction keyboard CDU? 

1/1 Yes 
5/5 No 

16. Are there any other training problems apparent for the multifunction keyboard CDU? 

0/1 Yes 
4/5 No 
2/0 Other: 

Not suitable to a two-man cockpit. (2) 

Learning the ABCs! (7) 

COMMENT: 

Subject 8 (U): As with any new system of this type the training concentrates far too much on 
operating the “new toy” and the basic operation of the DK takes a backseat. The basic opera- 
tion of the DK should be comfortable first and the “new toy” brought in. 
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GENERAL CONSIDERATIONS 


17. For the tasks performed today, which unit do you prefer to use? 

2/4 Much prefer the dedicated keyboard CDU 
3/0 Slightly prefer the dedicated keyboard CDU 
Like both equally 

1/0 Slightly prefer the multifunction keyboard CDU 
0/2 Much prefer the multifunction keyboard CDU 

Please explain your choice: 

Subject 1 (F): Ease of entry with DAKs. 

Subject 2 (F): MFK too slow to operate and requires finding key not in logical order. 

Subject 3 (U): The presentation and size of keys made the MFK unit seem less cluttered. 

Subject 4 (U): Ibo many keystrokes on the MFK! Both in choosing functions and inputting 
data! 

Subject 5 (F): Primarily because of alphanumeric work requirement on the MFK and previous 
experience (bias) with the dedicated CDU. 

Subject 6 (U): Much easier to select appropriate keys on MFK. A simpler display with easier 
logic and faster alpha key selection. It strikes me as a “new improved” version of the dedi- 
cated unit. While two steps are required, I nonetheless preferred it overall. 

Subject 7 (F): I’ve been using it for years. 

Subject 9 (F): More experience using the dedicated keyboard. 

Subject 10 (F): I’m much more familiar with the DK’s operations and expect changes as they 
occur. 

Subject 12 (U): DK requires less mental effort. 

18. Do you think the simulation was adequate to present the flying tasks? 

3/4 Yes 
2/1 No 
1/1 Other: 

Subject 3 (U): The entries to the CDUs were good. The actual flying while using the CDUs was 
not adequate due to the location of the CDU. 

Subject 5 (F): Normally, the copilot would be programming. Also, some questions require more 
experience with the units and a prior orientation to example questions to follow. 
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COMMENT: 


Subject 4 (U): The three modes were good choices. Flight tasks were none (preflight), low (CWS 
climb), and high (hand flying in turbulence). The simulation featured high breakout forces and 
displacements, with poor aircraft static stability. This provided an excellent task loading for 
the test. 

Subject 8 (U): No, full simulation should be used and fly a segment without the system, and fly 
it with the system to give a more realistic evaluation. 

Subject 10 (F): No, normally the pilot flying does not make FMC changes. 

Subject 11 (U): Yes. It would, however, be a good idea to familiarize the test subject with the 
units, and then let the test subject program and change the display as need be, instead of 
reading a step by step. 

19. Please use the space below to write any additional comments you might have for the 
dedicated keyboard CDU and the multifunction keyboard CDU. 

Subject 1 (F): Multientry keys increase “heads down” time in the cockpit. Procedures require a 
pilot to search for correct entry keys (i.e., to select the single letter, press, then select multi- 
choice). 

Subject 2 (F): I see no improvement to cockpit safety or crew efficiency in the MFK. 

Subject 3 (U): I felt there was a problem with using the line select keys on both unit. The MFK 
CDU seemed to be a bit more of a problem. 

Subject 4 (U): The MFK badly needs 26 to 28 keys. As it is, it is a threat to flight safety. I am 
not used to any CDU at all. I have a feeling that with practice the dedicated keyboard in- 
creases in superiority because a pilot memorizes key positions through usage. I am sure that 
data entry speed is probably twice as high as that using the MFK once experience is gained. 
This means the dedicated keyboard is safer because it distracts the pilot less. Unless the MFK 
adopts 26 to 28 keys, it can’t compare. 

Subject 5 (F): The dedicated keys on the multifunction CDU were not labeled or used. Perhaps 
using a combination of multi and dedicated keys combined on the same display unit would 
prompt different comments. I liked the large and very legible keys on the multifunction CDU. 

Subject 6 (U): I feel that the MFK would be better than the DK during turbulence and low 
light conditions. I suspect that locating the keyboard 2 or 3 inches aft on its location and 
pitched upward at, perhaps, 10 to 15 degrees, would facilitate its use. I found myself tending 
to lean forward to operate it. Why would I not then just move the seat forward? Because then 
I’d be improperly seated Tor flying the aircraft. 

Subject 7 (F): Needs a little work, but I think I could grow to like it. 

Subject 11 (U): Checklist. 
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1.0 INTRODUCTION 


1.1 Manual Description 

This training manual is to be used as part of training and familiarization for pilots participat- 
ing in the Flight Management Computer (FMC) Control Display Unit (CDU) test. It provides 
you with information about the dedicated keyboard CDU (737-300 FMC) and the multifunc- 
tion keyboard CDU concept. Read over this manual to familiarize yourself with the two CDU 
concepts. An instructor pilot will show the CDU operation to you prior to the test, and you will 
be given as much time as is needed to practice using both the dedicated keyboard and multi- 
function keyboard CDUs. 

This manual is divided into four sections; the first section describes the purpose for the test 
and serves as an introduction to multifunction and dedicated keyboards. The second section 
describes the dedicated keyboard and the multifunction keyboard to be used in this test. Key- 
board operations for both concepts are described in detail. The third section describes the 
laboratory setup and the data collection process. The fourth section describes a typical test 
session. 


1.2 Test Purpose 

The flight management computer CDU test is being conducted to compare two types of input 
devices: a dedicated keyboard and a multifunction keyboard. The task that will be used in this 
test for evaluation purposes is operation of the flight management computer (FMC) for the 
Boeing 737-300. The FMC control display unit (CDU) is the interface between the crew and 
the FMC. You will be asked to perform the same tasks on the FMC CDU configured with a 
multifunction keyboard and with the currently used 737-300 dedicated keyboard. 

A dedicated keyboard has keys, or switches, that always perform the same function. Activa- 
tion of a given key will always give the same results. A multifunction keyboard has keys that 
are capable of changing their function. The current function of a particular key is displayed on 
its programmable legend keyface. It requires fewer keys than the dedicated keyboard to ac- 
complish the same functions. The multifunction keyboard is an attempt to reduce the crew’s 
workload and manage the information flow by restricting the information presented to only 
that which is relevant to the current task or operation, while having other information avail- 
able on request. This data management could reduce the clutter and cross-checking problems 
that can occur when unnecessary information is combined with that which is presently 
required. 

This test will investigate the use of a multifunction keyboard in the place of the currently 
used dedicated keyboard for the operation of the 737 FMC. Using the Flight Deck Integration 
Research cab to allow simulation of flying tasks, a set of operations will be performed on the 
FMC CDU with both the multifunction keyboard and the dedicated keyboard. A brief ques- 
tionnaire will be administered after the session with each keyboard type, and a program 
questionnaire will be filled out at the end of the test sessions. Data will be recorded automati- 
cally during the session. £ 

fjp , , ; 
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2.0 DEDICATED KEYBOARD AND MULTIFUNCTION KEYBOARD CDUs 


2.1 Dedicated Keyboard CDU 

The dedicated keyboard CDU to be used in this test as shown in Figure B-l is currently used 
for the 737-300 FMC. It is divided into a keyboard section and a display section. The CDU 
data displays are shown on a 5-inch, cathode-ray tube (CRT). The general format of the CRT 
display is also given in Figure B-l. The basic display format consists of an array of 14 
24-character lines. A variety of alphanumeric characters and symbols can be presented on 
each line. The basic display format is subdivided by specifically defined display data blocks 
that are reserved in all displays for certain types of usage. 

The first five character spaces in the first line of the display are reserved for a data status 
indicator. This is a prefix to the page identifying title, which indicates whether the informa- 
tion currently on the display is the active executed flight plan (ACT) or a modification (MOD) 
to the active plan. If this data status block is blank, the data on the display is either not plan 
specific or not active. 

When flight plan data is entered, it needs to be activated and executed before it becomes an 
active route. When the system has an active route, all of the displays associated with that 
route have an “ACT” prompt in the title line of the page. Modifications in the data status area 
to any route plan can be made at any time. If changes are made to an active route, the “ACT” 
will be changed to “MOD.” The “MOD” designation goes away when the modifications are 
erased or when the modifications are executed, and “ACT” reappears in the display title line. 

The next 14 character spaces of the first line of the display are reserved for an identifying title 
for the display. The title may be a simple one that identifies the type of information presented 
in a data display (PROGRESS,' for instance), or it may be more complex, perhaps identifying a 
display which corresponds to a particular flight phase and the mode of operation in that phase 
at the same time (EC ON CRZ CLB). 

The remaining five character spaces in the first line are reserved for the page number for each 
display. This tells the operator the position of the current display in a string of pages that 
might exist for some FMCS functions. 

The entire 24 character spaces at the bottom (the 14th line) of the display are used as a scratch 
pad block for entry of data from the keyboard, or for display of failure messages, operational 
alerts, and advisory prompts. 

The remainder of the display (12 24-character lines) is available for use in displaying data 
without restriction. This data can consist of information directly applicable to operation of the 
aircraft, a menu allowing selection of system functional options, or a combination of the two. 

The data area is partitioned into specific data fields that are dependent on the display page. 
Entered data must be in the correct format, or else entry cannot occur. There are situations 
where certain data is required (e.g., the destination on the route page). This is indicated by the 
use of box prompts. If prompts are present on a display they must be filled in before you can 
go on. 
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2.1.1 Alphanumeric Key Group 

The CDU alphanumeric keyboard provides the crew with a means of entering data into the 
system for initialization of the navigation and guidance modes, and for modification of flight 
plans. All entries initiated by operation of the alphanumeric keyboard initially appear in the 
scratch pad area of the display. The scratch pad contents can be edited or cleared by pressing 
the CLR key. When an entry is present in the scratch pad, a brief depression of the CLR key 
will clear the last character in the entry. If the CLR key is held down for more than 1 second, 
the entire entry in the scratch pad will be cleared. 

The alphanumeric key group also includes the special-purpose slash (/), change sign (+/-), 
decimal point (.), and clear (CLR) keys. The slash key serves to separate data fields where 
multivariable entries are employed. An example of this type of entry is the definition of a 
climb speed schedule target airspeed and mach number (300/.780). The decimal point key 
inserts a decimal point into the scratch pad character string when pressed. The change sign 
key alternately changes the entry in the scratch pad from a positive to a negative quantity 
and back again. Repeated depressions of the change sign key simply toggle the positive or 
negative sense of the scratch pad entry. 

2.1.2 Flight Phase and Function Keys 

This group of 13 keys are the flight planning and flight phase keys. These keys provide the 
means for crew access to the flight planning functions of the FMCS. The specifics of flight 
phases (CLB, CRZ, and DES) can also be selected, once initially addressed, for display by using 
keys of this group. 

The phases of flight that can be preplanned and controlled by the FMCS are climb, cruise, and 
descent. The keys used to access the flight mode displays are: 

CLB (climb)— Displays current or alternate climb mode for assessment and selection. Cruise 
altitude (CRZ ALT) is enterable, as are speed/altitude restrictions. 

CRZ (Cruise)— Displays current or alternate cruise mode for assessment and selection. Infor- 
mation about optimum altitude-climb savings, and turbulence penetration Nl targets is also 
available. 

DES (descent)— Displays current or alternate descent mode for assessment and selection. End- 
of-descent (E/D) altitude is displayed and the airport-related speed/altitude restrictions are 
enterable. Flightpath angle (FPA), vertical speed (V/S), and vertical bearing (V/B) information 
is provided for crew reference. 

The flight planning keys are: 

INIT/REF (initialization/reference)— Allows access to data pages required for start-up of the 
FMCS and IRS. Also, the operator may select various reference data and maintenance pages. 

Nl LIMIT— Permits manual selection of the active Nl limit, and selection of any Climb Nl 
Limit thrust reduction that may apply. 
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DEP/ARR (departures/arrivals)— Used for selection of the procedures and runways at the ori- 
gin and destination airports entered by the crew. 

RTE (route)— Gives access to flight plan data selections and entries into the FMC in clearance 
language. This will probably be the primary key for entry of lateral flight plan information. 

LEGS (route legs)— Displays and accepts entries of detailed data concerning each leg of the 
flight plan for both the lateral and vertical paths. 

HOLD— Permits planning or initiation of holding at a designated waypoint. 

FIX (Fix information)— Displays range and bearing data from the present position to an 
entered Fix. 

PROG (flight progress)— Displays current flight status information such as ETA, fuel remain- 
ing at waypoint, navigation radio tuning status, wind, and path errors. 

The remaining key of this category is the EXEC (execute) key. This key is used to incorporate 
displayed data in the FMC as part of the active flight plan. The EXEC key is operable when 
its annunciator bar is illuminated. The key is used for activating the flight plan, changing the 
active flight plan, changing the active guidance mode, or inserting data that affect the active 
flight plan, guidance mode, or data base. Illumination of the white annunciator bar indicates 
that a valid set of data is on display and may be made active for guidance of the aircraft. 

2.1.3 Page Select Keys 

If a function has more than one page of display data associated with it, the page number block 
at the upper right corner of the CRT display will indicate this. The page number actually 
consists of two numbers separated by a slash. The First number indicates what page is cur- 
rently displayed, and the second number indicates total number of pages in the string. Press- 
ing the NEXT PAGE key advances the display to the next higher numbered page, while 
pressing the PREV PAGE key backs up the display to the next lower numbered page. The 
page strings are continuous, however, in that advancement past the last page in a string 
(beyond 9/9, for example) would take the display back to the first page in the string (1/9, in 
this case). The same sort of thing occurs if the first page of a string is displayed and the PREV 
PAGE key is pressed (i.e., the display shows the last page in the string). 

2.1.4 Line Select Keys and Scratch Pad Usage 

Twelve line select keys are provided on the CDU panel, as shown in Figure B-l, six each to the 
left and right of the display screen. The line select keys perform three functions. They can be 
used to transfer information from the scratch pad area up into the appropriate display area. 
They can be used to copy information from the display to the scratch pad. Finally, they can be 
used to designate a chosen item in a displayed list (e.g., selecting a specific runway from a list 
of candidate runways.) These functions are explained below. 

All manual data entries appear initially in the scratch pad area. In the scratch pad, they can 
be verified and, if necessary, edited. Entry of the data into the operational display is accom- 
plished by pressing an appropriate line select key. 
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In addition to transferring data from the scratch pad up to the data area of the display, the 
line select keys can be used to duplicate data from a data area line into the scratch pad. This is 
done when the scratch pad is blank, simply by pressing the line select keys associated with the 
data desired for duplication in the scratch pad. The data appears in the scratch pad and, by 
use of the line select keys may then be transferred up to another data line of the display. If the 
display page is changed while there is an item in the scratch pad, the contents of the scratch 
pad will carry over and can then be inserted into any data line on the new page for which it is 
valid. 

At certain times during the FMC CDU operation, the data area will show several options from 
which the pilot can choose. An example is a list of runways for a given departure airport. A 
depression of the line select key next to a runway will designate that runway as chosen. 

2.1.5 Delete Key 

The delete (DEL) key is a special purpose key that is used to remove data from a display (and 
thus a flight plan) after it has been line selected and accepted into a data field. If the scratch 
pad is empty, pressing the DEL key will write “DELETE” into the scratch pad. The delete 
process is then completed by the operator’s line selecting the data item he desires removed. If 
the deletion is a valid one, the data field reverts to its default value (box prompts, dashes, or a 
system-generated value). When use of the DEL key would be incorrect, the system will pre- 
vent the operator from making mistakes. Selection of the wrong line select key on a page in 
which the delete operation is valid for at least one display parameter brings up an INVALD 
DELETE message in the scratch pad. Pressing the DEL key when the displayed page has no 
valid delete operation evokes no response. The DEL key must be pressed once for each param- 
eter to be deleted. 


2.2 Multifunction Keyboard CDU 

The multifunction keyboard CDU is shown in Figure B-2. As can be seen, the CDU is divided, 
like the dedicated keyboard, into the display section and the keyboard section. The display 
and the pages shown during the operation are identical to the dedicated keyboard CDU. The 
keyboard area is very different from the 737 CDU, as there are only 15 keys on the multifunc- 
tion CDU versus over 50 on the dedicated keyboard CDU. The functions displayed on the 
multifunction keys will, in this test, virtually be identical to the 737 FMC dedicated keyboard. 
Only selected functions will be available at any one time, however. The keyboard configura- 
tion shown in Figure B-2 is the top level configuration. It comprises the flight phase and 
function keys, and the step back key. Selection of any key will cause the keyboard to 
reconfigure. 

Three of the four dedicated keys immediately below the display area will be used during this 
test. They are the CLEAR, NEXT PAGE, and PREV PAGE keys. The CLEAR key affects only 
contents of the scratch pad line and operates the same as on the dedicated keyboard. A brief 
press of the CLEAR clears the last-entered character and moves the data entry marker back 
one space. A longer press of the CLEAR key clears the entire scratch pad. NEXT PAGE 
advances one page as described for the dedicated keyboard. PREV PAGE goes back one page, 
as described for the dedicated keyboard. 
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Figure B-2. Multifunction Keyboard CDU (Top Level Keyboard Configuration) 






The multifunction keys are software controlled to reformat (change legend and function) in 
accordance with previous operation(s). These keys may reformat when any of the multifunc- 
tion keys are operated, or when line select keys are operated. The key legends and associated 
functions described are below. 

a) The numeric keys enter the selected number, decimal (.), or sign (+/-) into the position 
in the scratch pad indicated by the data entry marker. 

b) The alphabetic keys require two operations for alpha entry. The first alphabetic key- 
board has three letters per key, similar to the telephone keyboard arrangement 
(ABC, DEF, etc.). Selection of a letter set will result in a reformat of the keys in that 
row to display the three letters (i.e., A, B, and C, on individual keys). The individual 
alpha keys enter the selected letter into the position in the scratch pad indicated by 
the data entry marker. 

c) The TOP LEVEL key returns the keyboard to the highest logic level. Any provisional 
entered data will be deleted when this key is selected unless it is recalled by using 
the STEP BACK key on the next key stroke. 

d) The STEP BACK key negates the previous key operation. 

e) The EXECUTE key is the “command” key and operates like the EXECUTE key on 
the dedicated keyboard. When provisional data is ready for entry, the EXECUTE key 
will become available to send the data to the flight management computer. 

f) Actuation of a flight phase and function key causes display of the first (or only) page 
in that mode. Flight phase and function keys are identical to those for the dedicated 
keyboard CDU. 

g) The index key returns the keyboard to the highest level configuration for a particular 
* mode (i.e., one level down from the top level configuration). 

Most of the FMCDU modes allow data entry. Entries are performed in the same manner as for 
the dedicated keyboard CDU, using the scratch pad area and line select keys. Alpha or nu- 
meric entries are made by selecting the appropriate alpha and numeric keys. After the new 
line is in the scratch pad, it is entered in the data line by a press of the line select key. The 
data must be a valid entry with the correct format before entry will occur. 

Hands-on training prior to testing will be given on the dedicated keyboard and the multifunc- 
tion keyboard CDUs described above. This will allow you to see how the CDUs actually 
operate. 
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3*0 LABORATORY SETUP, DATA COLLECTION, AND TEST SCENARIOS 


3.1 Laboratory Setup 

The FMC CDU test will be conducted in the Renton Simulation Center with the research cab 
as the test vehicle. The research cab does not represent a specific aircraft configuration, but 
will be running software to simulate the 737-300. The aircraft model is being used solely to 
provide a flight task and should not be evaluated with respect to actual flight characteristics. 

The test setup will accommodate both the multifunction keyboard CDU and the dedicated 
keyboard CDU. You will use both configurations during the course of the test. Flying tasks 
will be performed concurrently with the operation of one of the CDUs. This will allow for a 
realistic comparison of the two CDUs. 

3.2 Data Collection 

Data collected during the session will include flight performance data, eye view monitor data, 
and subjective data. Flight performance data includes several parameters automatically re- 
corded during the flying tasks. These will be specifically related to flying the airplane (e.g., 
altitude deviations). 

A second method of collecting data involves the eye view monitor. The eye view monitor is a 
device that collects real time data on where you are looking. It consists of a tracking mirror 
mounted on the instrument panel, a head tracking module, and a camera unit that feeds 
information into the computer. The head tracking module consists of a headband with a light 
weight unit attached. You will be asked to wear this during the test. The eye view monitor is a 
noninterference unit; care has been taken to assure that its use will be comfortable and will 
not distract you from performance of the tasks. * 

The third data collection device is pilot questionnaires. A brief questionnaire will be filled out 
after each CDU is used. A program questionnaire will be the final step in the testing process. 
You will be asked to provide your comments and impressions about the two keyboard configu- 
rations via the questionnaires. 


3,3 Test Scenarios 

Six scenarios will be performed on each CDU. Two scenarios, Flight Plan Route Entry and 
Performance Initialization, will be performed preflight. Modify Route and Enter Altitude Re- 
striction will be performed during cruise. During the approach phase, Set Up Approach and 
Time/Distance to Crossing Radial will be performed. Appendix C lists the six scenarios, and 
presents the general steps in each scenario. The exact sequence of key presses will differ 
between the dedicated keyboard CDU and the multifunction keyboard CDU. You do not need 
to memorize the specific entries in each scenario; an instructor pilot will be present during the 
testing to give you the necessary information. 
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4.0 TEST PROCEDURE 


A typical test session will be conducted as described below. 

Upon arrival at the test location, time will be taken to explain the test setup and data collec- 
tion process. You will be given an introduction to the simulator and have an opportunity to 
become familiar with flying the airplane. You will be positioned at the eye reference point in 
the simulator. The eye view monitor will be calibrated at this time, which will allow eye 
tracking data to be recorded during the subsequent testing. 

A period of familiarization with the first CDU (either the dedicated keyboard or the multifunc- 
tion keyboard) will be conducted. This time can be used to practice using the CDU. Any 
questions you have about the CDU or about how the test will be run can be answered at this 
time. An instructor pilot will show each scenario. You can run through the scenarios until you 
are able to complete them with no errors. The instructor pilot will be present during training 
and during the test to tell you each specific entry (e.g., enter KYKM for destination). After you 
practice operating the CDU while performing the flying tasks, the next step will be a test of 
the first CDU. This will consist of performing the six scenarios listed in Appendix C; two 
operations during three flight phases. 

The scenarios will be presented in random order, and you will be informed of which flight 
phase you are in prior to each test trial. Each operation will last several minutes. At the 
beginning of each operation, the simulation will be started. A period of time will be given to 
allow stable flying (except for those two scenarios performed during preflight). When you are 
comfortable with flying, the CDU operation will be performed. At the end of the CDU opera- 
tion the simulation will be stopped. Data will be collected automatically on the flight parame- 
ters and with the eye view monitor. 

A brief questionnaire on the first CDU will be given to you when all six scenarios are com- 
pleted. This questionnaire will cover features of the first CDU to elicit a subjective evaluation 
of the CDU operation. Any observations, suggestions, and comments that you might have will 
be appreciated. 

After a short break to change CDU configuration, a familiarization period will be conducted 
with the second CDU. A test on the second CDU will be run next, and will consist of the same 
six scenarios as the first CDU. A brief questionnaire on the second CDU will be given to you, 
followed by a program questionnaire. The program questionnaire is longer than the others, 
and contains comparative questions about the dedicated keyboard and multifunction keyboard 
CDUs. 

The entire session is expected to last approximately four hours. 
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APPENDIX C 


TRAINING AND TEST SCENARIOS 



TEST SCENARIO 1: ENTER FLIGHT PLAN 


Flight Phase— Preflight 


Go to ROUTE page . 

Enter KB FI into "ORIGIN" 

Enter KMWH into "DEST" 

Enter RW13R into "RUNWAY" 

Enter ELN for first waypoint 
Enter EPH for second waypoint 
Activate 
Execute 

Go to KB F I DEPARTURE page 
Select LACRE3 for "SID" 

Select VAMPS for "TRANS" 

Execute 

Go to LEGS page 

Line Select ELN into "THEN" 

Execute 

Verbal instructions to pilot: 

Positioned on Runway 13R at Boeing Field prior to fiignt plan 


entry . 



TEST SCENARIO 2: PERFORMANCE INITIALIZATION 


Flight Phase— Preflight 


Go to I NIT REF page 
Select INDEX 
Go to PERF I NIT page 
Enter 110 into "Gross wt 
Enter 5 into "Reserves" 

Enter 3b into "COST INDEX 1 
Enter 18000 into "TRP/CRZ ALT" 

Execute 

Enter 270/55 into "CRZ WIND" 

Enter -5 INTO "ISA DEV" 

Execute 

Verify transition altitude 
Verbal instructions to pilot: 

Position is on Runway 13R at Boeing Field prior to performance 
initialization. 
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TEST SCENARIO 3: MODIFY ROUTE 


Flight Phase— Climb 


Go to ROUTE page 
Enter KYKM into "Dest" 

Execute 

Go to LEGS page 

Enter HUMPP for first waypoint 
Enter RADDY for second waypoint 
Execute 

Go to KYKM ARRIVAL page 
Select ILS 27 for "Approaches" 

Select SUNED for "Trans" 

Execute 

Verbal instructions to pilot: 

Position is 4 miles from VAMPS, 7UUu ft. and climbing at 2bU 
knots. Route modification consisting of new destination. 
Climb out at 2b0 knots using CWS.; maintain constant airspeed 
and heading. 
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TEST SCENARIO 4: ENTER ALTITUDE RESTRICTION 


Flight Phase— Climb 


Go to LEGS page 

Enter 7000B into VAMPS waypoint 
Enter 210/ into LACRE 
Execute 


Verbal instructions to pilot: 

Position is 400 ft. and climbing just after liftoff 
Speed is 200 knots. Speed and altitude restriction 
Climb out at 200 knots and flaps 1 using CWS. 
constant speed and heading. 


from KBFi. 
received . 
Mai ntai n 



TEST SCENARIO 5: SETUP APPROACH 


Flight Phase— Top of Descent 

Go to KMWH ARRIVAL page 
Select ILS 32R for "APPRUACH" 

Select MOSES 1 for "STAR" 

Execute 

Go to LEGS page 

Line select CF32R into "THEN" 

Execute 


Verbal instructions to pilot: 

Position is approximately 10 miles prior to top of descent in 
moderate turbulence. Speed 320 knots, altitude 18,000 ft. 
Approach clearance received. Maintain heading and 18,000 tt. 
al titude . 
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TEST SCENARIO 6: TIME/DISTANCE TO CROSSING RADIAL 
Flight Phase— Top of Descent 

Go to FI X page 
Enter MWH into "Fix" 

Enter 120/ into "DNTKFX" 

Bring second line of information into scratchpad 

Go to LEGS page 

Enter scratchpad into "THEN" 

Line Select CF32R into "THEN" 

Execute 

Verbal instructions to pilot: 

Position is approximately lu miles prior to top of descent in 
moderate turbulence. 320 knots, altitude 18,000 ft. Time and 
distance to the crossing of the MWH 12U radial requested. 
Maintain heading and level flight. 
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TRAINING SCENARIO 7: ENTER FLIGHT PLAN 
Flight Phase— Preflight 

Go to ROUTE page . 

Enter KSEA into “ORIGIN" 

Enter KPDX into "DEST" 

Enter RW16R into "RUNWAY" 

Enter PUW for first waypoint 
Enter ALW for second waypoint 
Activate 
Execute 

Go to KSEA DEPARTURE page 
Select MOUNT 1 for "SID" 

Select GEG for "TRANS" 

Execute 

Go to LEGS page 
Select PUW into- "THEN" 


Execute 



TRAINING SCENARIO 8: PERFORMANCE INITIALIZATION 
Flight Phase— Preflight 

Go to I N IT REF key 
Select INDEX 
Go to PERF I N IT page 
Enter 130 into "Gross wt 
Enter 3 into "Reserves" 

Enter 40 into "COST INDEX' 

Enter 25000 into "TRP/CRZ ALT" 

Execute 

Enter 185/30 into "CRZ WIND" 

Enter 7 INTO "ISA DEV" 

Execute 


Verify transition altitude 



TRAINING SCENARIO 9: MODIFY ROUTE 


Flight Phase— Climb 

Go to ROUTE page 
Enter KB 0 1 into "Dest" 

Execute 

Go to LEGS page 

Enter LWS for first waypoint 

Enter MYL for second waypoint 

Execute 

Go to KB 0 1 ARRIVAL page 
Select ILS 10R for "Approaches" 

Select PARMO for "Trans" 

Execute 

TRAINING SCENARIO 10: ENTER ALTITUDE RESTRICTION 
Flight Phase — Climb 

Go to LEGS page 

Enter 18UU0B into ELN waypoint 

Enter 290/ into ELN 

Execute 
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TRAINING SCENARIO 11: SET UP APPROACH 


Flight Phase— Top of Descent 

Go to KPDX ARRIVAL page 
Select ILS 1UR for "APPROACH" 

Select DLS 1 for "STAR" 

Execute 

Go to LEGS page 

Line select SAUVI- into "THEN" 

Execute 


TRAINING SCENARIO 12: TIME/DISTANCE TO CROSSING RADIAL 
Flight Phase— Top of Descent 

Go to FIX page 
. Enter PDT into "Fix" 

Enter 350/ into "DNTKFX" 

Bring second line of information into scratch pad 

Go to LEGS page 

Enter scratchpad into "Then" 


Execute 



"?’X' * 



APPENDIX D 


TLA OUTPUT 


PRECEDING PAGE BLANK NOT FILMED 



MISSION TIMELINE 
MISSION - FROM SCEN.3A - ILS 
WITH 15-KEV CDU 


CONFIGURATION - NASA 515 -AFD 
FLIGHT PHASE - CRUISE - AREA NAV 
COMPARE CDU IS (EXP) 
15-KEV MULTI-FUNC 


MAR 5» 198b 


CREWMEMBER - CO-PILOT 


CODE 

EVENT/PROCEDURE OR 
TASK DESCRIPTION 

TASK DUR 
(SEC) 

TIME IN SECONDS 

19X001 
8AX 1 

MON. EXT. VIS. 

47.00 

7 

u 

130010 

SET UP NEW 4-D 


V 

02X030 

FLIGHT PLAN 
SELECT ATC CLEARANCE 


V 

2LX 109 

PAGE 

PRESS TOP LEVEL 

1.68 

0 

2LX 81 

KEV (X) 

PRESS FLT PLAN 

1.77 

D 

2LX 111 

KEV (X) 

PRESS ATC CLEARANCE 

1.77 

Q 

2L 08 

KEV (X) 

MON ATC CLR MODE 

2.34 

0 

13X018 

DATA 

DESTINATION NAME - 


V 

2LX 160 

LINE SELECT ORIGIN 

1.83 

0 

2LX 161 

(X) 

line select 

1.83 

' 0 

2L 08 

DESTINATION (XI 
MON ATC CLR MODE 

2.34 

0 

13X034 

DATA 

ATC CLR PAGE - ROUTE 


V 

2LX 162 

INPUT 

LINE SELECT V/A (X) 

1 

1.83 

0 

2LX 163 

PRESS RNAV KEV (X) 

1.77 

Q 

2LX 175 

LINE SELECT J-815R 

1.83 

0 

2LX 164 

TO SP 

LINE SELECT J-815R 
(X) 

MON ATC CLR MODE 

1.83 

0 

p 

o 

CO 

2.34 

□ 

13X035 

DATA 

ATC CLR PAGE - TO 


V 

2LX 176 

INPUT 

LINE SELECT SHI 01 

1.83 

0 

2LX 165 

TO SP 

LINE SELECT TO (X) 

1.83 

D 

i i i— - i 1 1 1 1 i- 


0.0 50.0 100. 150. 200. 250. 


Figure D-1 . Mission Timeline — Multifunction Keyboerd, Cruise 
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PAGE a 



EVENT /PROCEDURE OR 

TASK DUR 




CODE 

TASK DESCRIPTION 

(SEC) 


TINE IN SECONDS 


2LX 120 

PRESS GRP 
KEV CX) 

1.77 


0 


2LX 166 

LINE SELECT NACEV 
(X) 

1.83 


0 


2L 08 

NON ATC CLR NODE 
DATA 

2.34 


0 


13X032 

ARRIVALS PAGE - STAR 
INPUT 



V 


p» 

X 

o 

PRESS ARRIVAL 
KEV (X) 

1.S2 


0 


2LX 167 

LINE SELECT NACEV2 

1.83 


0 


1,3X033 

(X) 

LEGS 1 PAGE - PTA 
INPUT 



V 

9 

2LX 99 

PRESS LEGS 1 
KEV (X) 

1.S2 


D 


2LX 168 

LINE SELECT LAKESIDE 
PTA (X) 

1.83 


0 


2LX 26 

PRESS NO. 1 KEV (X) 

1.18 


0 


2LX 3S 

PRESS NO. 0 KEV (X) 

0.83 


1 


2LX 27 

PRESS NO. 2 KEV (X) 

0.83 


1 


2LX 26 

PRESS NO. 1 KEV (X) 

0.83 


1 


2LX 168 

LINE SELECT LAKESIDE 
PTA (X) 

1.83 


0 


02X043 

REVIEU PROVISIONAL 
FLT PLAN DATA 



V 


2L 09 

NON FLT PLN 1 NODE 
DATA 

2.08 


□ 


2K 17 

NON NAP VIDEO 

2.27 


0 


2LX 169 

LINE SELECT STEP KEV 
(X) 

1.83 


0 


02X043 

REVIEU PROVISIONAL 
FLT PLAN DATA 



V 


2L 09 

NON FLT PLN 1 NODE 
DATA 

2.08 


D 


2K 17 

NON NAP VIDEO 

2.27 


□ 


2LX 169 ; 

LINE SELECT STEP KEV 
(X) 

1.83 


0 


02X043 

REVIEU PROVISIONAL 
FLT PLAN DATA 



V 


2L 09 

NON FLT PLN 1 NODE 
DATA 

2.08 


0 


2K 17 

NON NAP VIDEO 

2.27 


0 


2LX 169 

LINE SELECT STEP KEV 
CX) 

1.83 


D 


020044 

ACCEPT PROVISIONAL 
FLIGHT PLAN 


1 L 

V 

1 1 1 -J L 



0.0 50.0 100. 150. 200. 250. 


Figure D-1. Mission Timeline— Multifunction Keyboard, Cruise (Continued) 
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PAGE 3 


- CODE 

EVENT/PROCEDURE OR 
TASK DESCRIPTION 

TASK DUR 
(SEC) 

TIME IN SECONDS 

2L 21 

PRESS EXEC KEV 

1.52 

8 

2L 09 

NON FLT PLN 1 MODE 

2.08 

0 


DATA 



2K 17 

MON MAP VIDEO 

2.27 

D 

IPO 20007 

CALL OUT- [FLIGHT 

2.00 

0 


PLAN ENTERED AND 




CHECKED] 



02X044 

FL INPUT - 320 


V 

2LX 109 

PRESS TOP LEVEL 

1.68 

0 


KEV (X) 



2LX 80 

PRESS PERF DATA 

1.77 

D 


KEV (XI 



2LX 94 

PRESS DESCEND 

1.77 

0 


KEV (XI 



2L 09 

MON FLT PLN 1 MODE 

2.34 

0 


DATA 



2LX 170 

LINE SELECT TARGET 

1.83 

0 


ALT KEV (X) 



2LX 140 

PRESS FL 

1.77 

0 


KEV (X) 



2LX 28 

PRESS NO. 3 KEV (X) 

0.83 

1 

2LX 27 

PRESS NO. 2 KEV (X) 

0.83 

1 

2LX 35 

PRESS NO. 0 KEV (X) 

0.83 

1 

02X045 

GS INPUT - 250 


V 

2LX 170 

LINE SELECT TARGET 

1.83 

0 


ALT KEV (X) 



2L 09 

MON FLT PLN 1 MODE 

2.34 

D 


DATA 



2LX 171 

LINE SELECT CMO 

1.83 

0 ' 


SPEED (X) 



2LX 27 

PRESS NO. 2 KEV (X) 

1. 18 

0 

2LX 30 

PRESS NO. 5 KEV (X) 

0.83 

1 

2LX 35 

PRESS NO. 0 KEV (X) 

0.83 

1 

2LX 171 

LINE SELECT CMD 

1.83 

0 

' 

SPEED (X) 


l l 1 1 — J i J 1— 


0.0 50.0 100. 150. 200 . 250. 


Figure D-1. Mission Timeline— Multifunction Keyboard, Cruise (Concluded) 
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1 SHIFTED 1 


MISSION TIMELINE 
MISSION - FROM SCEN.3A - ILS 
UITH 15-KEV COU 
CONFIGURATION - NASA SIS -AFD 
FLIGHT PHASE - DESCENT - AFD 

COMPARE COU IS (EXP) 
15-KEV MULTI-FUNC 
CREWMEMBER - CO-PILOT 


MAR S. 198b 




EVENT/PROCEDURE OR 

TASK DUR 





CODE 

TASK DESCRIPTION 

(SEC) 



TIME IN SECONDS 

I 150023 

RECEIVE INSTRUCTIONS 


7 



1R 

35 

TO CHANGE LAKESIDE 
PTA Tp 10 22 15 

8.00 




MON VHP -2 COMM AUDIO 




l P 1 SO 009 

MON RADIO COMM - 

4.00 

ID 





[NASA SIS, DUE TO 
TRAFFIC VOUR PLANNED 
TIME OF ARRIVAL AT 





1P150010 

LAKESIDE IS NOU 

4.00 

CD 





10 22 15 , OVER] 





1R 

39 

COMM VIA VHF-2 

4. SO 


□ 


IP ISO 0 1 1 

RADIO COMM - C515# 

4.50 


CD 




ROGER, TIME OF AR- 
RIVAL NOU 10 22 151 





1R 

30 

ACT PUSH-TO-TALK SU 
ON CONTROL HANDGRIP 

4. SO 


CD 


15X024 

REVISE FLIGHT PLAN 




V 



TO CHANGE LAKESIDE 
PTA 





2LX 

109 

PRESS TOP LEVEL 
KEV (X) 

1.68 



□ 

2LX 

81 

PRESS FLT PLAN 
KEV (X) 

1.77 



□ 

2L 

09 

MON FLT PLN 1 MODE 
DATA 

2.34 



□ 

2LX 

168 

LINE SELECT LAKESIDE 
PTA (X) 

1.83 



□ 

2LX 

26 

PRESS NO. 1 KEV (X) 

1.18 



0 

2LX 

35 

PRESS NO. 0 KEV (X) 

0.83 



0 

2LX 

27 

PRESS NO. 2 KEV (X) 

0.83 



a 

2LX 

27 

PRESS NO. 2 KEV (X) 

0.83 



0 

2LX 

26 

PRESS NO. 1 KEV (X) 

0.83 



D 

2LX 

30 

PRESS NO. 5 KEV (X) 

0.83 



0 

2LX 

168 

LINE SELECT LAKESIDE 
PTA (X) 

1.83 



□ 

2L 

09 

MON FLT PLN 1 MODE 
DATA 

2.34 

\ 

..I 

□ 


165. 185. 80S. 225. 245. 2bS. 


Figure D-2. Mission Timeline— Multifunction Keyboard , Descent 
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■ CODE 


EVENT/PROCEDURE OR 
TASK DESCRIPTION 


TASK DUR 
(SEC) 
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PAGE 2 



Keyboard , Descent (Concluded) 




1 UNSHIFTEP] 


MISSION TIMELINE 


MISSION - FRCM SCEN.3A - ILS 

DEE I GATED KEVBO. CDU 
CONFIGURATION - NASA SIS -AFD 
FLIGHT PHASE - CRUISE - AREA NAV 
COMPARE CtXJIS 
DEDICATED KEVBOARD 
CREWMEMBER - CO-PILOT 


mar 


1986 


cor.iE 

EVENT/PROCEDURE OR 
TASK DESCRIPTION 

TASK DUR 
(SEC) 

TINE IN SECONDS 

19X001 

MONITORING 

1 

47.00 

7 

8AX 1 

NON. EXT. VIS. 

” 1 

130010 

SET UP WEU 4-D 
FLIGHT PLAN 

*7 

000048 

SELECT RTF PG 


V 

2l 165 

PRESS ROUTE KEV 

1.48 

0 

0'.. 08 

MON CDU DISPLAV 

2.08 

n 

130030 

SE_ECT OR IG TO DEST 


V 

01- 175 

L.IME SEL - KEV 

1.83 

D 

2l 175 

LINE SEL - KEV 

1.83 

0 

2L 175 

LINE SEL - KEV 

1.83 

0 

8L 08 

MON COU DISPLAV 

2.08 

D 

2u 21 

PRESS EXEC KEV 

1.40 

fl 

2*. 03 

MON COU DISPLAV 

2.08 

r 

19X002 

MONITORING 


V 

8AX 2 

MON EXT VIS 

8,00 

; 

130031 

RTE STRUCTURE 
DESTINATION 


7 

2l 161 

PRESS DIR/IMIT KEV 

2.03 

G 

2u 44 

PRESS H/RTE KEV 

1.3S 

a 

2L 175 

LINE SEL - KEV 

1.83 

N 

2w 175 

LINE SEL - KEV 

1.83 

D 

2'- 44 

PRESS H/RTE KEV 

1.48 

0 

2l 175 

LINE SEL -- KEV 

1.83 

u 

2l 175 

LINE SEL - KEV 

1.83 

0 

130032 

ENTER FLT. LEVEL 
320 


7 

2l 23 

PRESS NO. 3 KEV 

1.3S 

0 

2i_* 27 

PRESS NO, 2 KEV 

1.35 

11 

2l 35 

PRESS NO. 0 KEV 

1.35 

0 

2l 175 

LINE SEL - KEV 

2.08 

D 

020051 

XEQ CHG 


V 

2l 08 

MON CDU DISPLAV 

2.08 

0 

2i_ 21 

PRESS EXEC KEV 

1.42 

G 

2L 08 

MON CDU DISPLAV 

2.08 

Q 

19X002 

MONITORING 


V 

8AX 2 

MON EXT VIS 

8.00 

□ 

i i i i i i i ; i 


0.0 50.0‘ 100. 150. 800. 850. 


Figure D-3. Mission Timeline— Dedicated Keyboard , Cruise 


C- ci_ 


97 










CODE 

EVENT /PROCEDURE OR 
TASK DESCRIPTION 

TASK DUR 
(SEC) 

020052 

ESTABLISH ARRIVAL 
INFO 


2' 52 

PRESS P/STAR KEV 

1.35 

21 175 

LINE SEL - KEV 

1.33 

2L 175 

LINE SEL - KEV 

1.83 

130033 

ENTER SPD. 


2L 27 

PRESS NO. 2 KEV 

1.35 

2L 30 

PRESS NO. 5 KEV 

1.35 

2u 35 

PRESS NO. 0 KEV 

1.35 

a. i7s 

LINE SEL - KEV 

2.08 

020051 

XEQ CHG 


2' 03 

NON CDU DISPLAV 

2.08 

2- 21 

PRESS EXEC KEV 

1.42 

2L 08 

NON CDU DISPLAV 

2.08 

19X002 

NON I TORINO 

8.00 

8AX 2 

NON EXT VIS 

020053 

ESTABLISH LAKESIDE 
LON ARRIVAL 
CLEARANCE 


2l 169 

PRESS DES KEV 

2.03 

2L 175 

LINE SEL - KEV 

1.83 

2L 26 

PRESS NO. 1 KEV 

1 . 35 

2L 35 

PRESS NO. 0 KEV 

1.35 

IPO 10071 

SION TO PUSHBACK. 

UE HAVE INFORMATION 
KILO. OVER] 

3.00 

2L 27 

PRESS NO. 2 KEV 

1.35 

2L 26 

PRESS NO. 1 KEV 

1.3S 

IQ 16 

NON VHF-1 COMM AUDIO 

6.00 

IPO 10072 

NON RADIO COMM- 
[NASA 515. ATLANTA 
GROUND. ROGER. CLEAR 
“0 PUSHBACK . ADVISE 

4.50 

2L 08 

NON CDU DISPLAV 

2.34 

2l 175 

LINE SEL - KEV 

1.83 

2L 175 

LINE SEL - KEV 

1.83 

IPO 10075 

UHEN READY TO TAXI. 
OVER.] 

1.50 

2l 175 

LINE SEL - KEV 

1.83 

IPO 10064 

RADIO CONTI- [NASA 
515. ROGER] 

1.70 

10 24 

ACTUATE PUSH-TO-TALK 
SU ON CONTROL HAND- 
GRIP 

1.70 

IQ 15 

COMM VIA VHF-1 

1.70 

2L 08 

NON CDU DISPLAV 

2.08 

2L 175 

LINE SEL - KEV 

i 

1.83 
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MISSION TIMELINE 
MISSION - FROM SCEN.3A - ILS 

DEDICATED KEVBD. CDU 
CONFIGURATION - NASA S15 -AFD 
FLIGHT PHASE - DESCENT - AFD 
COMPARE CDUS 
DEDICATED KEVBD. 
CREWMEMBER - CO-PILOT 


MAR S. 1986 
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PAGE 2 


CODE 

1P140039 


1R 16 
1P140040 
15002b 
1P140041 


1R 

24 

1R 

* 40 


EVENT/PROCEDURE OR TASK DUR 
TASK DESCRIPTION I SEC) 


MON RADIO COMM - 
[NASA 515, CONTACT 
APPROACH CONTROL ON 
ONE TUO SIX POINT 
MON VHF-2 COm AUDIO 
NINER, OVER] 

REACH 220 KIAS 
RADIO COMM -[NASA 
515, ROGER. APPROACH 
ON ONE TUO SIX POINT 
NINER] 

ACT PUSH-TO-TALK SU 
ON CONTROL HANDGRIP 
conn VIA VHF-2 


4.00 

5.00 

1.00 

4.00 

4.00 

4.00 


TIME IN SECONDS 


CD 


{=□ 

a 

v 


□ 


□ 

□ 


165. 185. 205. 225. 24$. 265. 


Figure D-4. Mission Timeline— Dedicated Keyboard, Descent (Concluded) 
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MISSION 

FROM SCEN.3A - ILS 
DEDICATED KEVBD. CDU 



























UNSHIFTED 


CHANNEL ACT I VI TV SUMMARY 
MISSION - FROM SCEN.3A - ILS 
WITH 1S-KEY CDU 


MAR 5. 


PERCENT WORKLOAD I NG AVERAG eL— 1 +1SIGMA 

0 20 40 60 80 100 

r r~”r~ — i — — i 1 f i i i i r 


EXTERNAL VISION 
INTERNAL VISION 
LEFT HAND 
RIGHT HAND 
LEFT FOOT 


RIGHT FOOT 


COGNITIVE 


a 


AUDITORV 


VERBAL 


J J— I 

FLIGHT PHASE CONFIGURATION 

CRUISE - AREA NAV NASA SIS -AFD 

COMPARE CDU IS (EXP) 

1S-KEV MULTI-FUNC 


v WEIGHTED CHANNEL 
AVERAGE 


L __1 — 

CREWMEMBER 

CO-PILOT 


Figure D-13. Channel Activity Summary — Multifunction Keyboard, Cruise 


1986 

BBB 

120 
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UNSHIFTED 


CHANNEL ACTIVITY SUMMARY 
MISSION - FROM SCEN.3A - ILS 
WITH 15-KEY CDU 


MAP 5. 1986 


PERCENT WORKLOADING AVERAGeCZH +1 SIGMA EE3 


EXTERNAL VISION 
INTERNAL VISION 


LEFT HAND 


RIGHT HAND 
LEFT FOOT 
RIGHT FOOT 
COGNITIVE 
AUDITORY 


VERBAL 


UEIGHTED CHANNEL 
AVERAGE 

FLIGHT PHASE 
DESCENT - AFD 
COMPARE CDU i S (EXP) 
15-KEY MULTI-FUNC 


0 20 40 60 80 100 120 



NASA 515 -AFD CO-PILOT 


Figure D-14. Channel Activity Summary— Multifunction Keyboard, Descent 
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MAR 5. 1986 


EXTERNAL VISION 
INTERNAL VISION 
LEFT HAND 
RIGHT HAND 
LEFT FOOT 
RIGHT FOOT 
COGNITIVE 
AUDITORY 
VERBAL 

WEIGHTED CHANNEL 
AVERAGE 

FLIGHT PHASE 
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Figure D-15. Channel Activity Summary— Dedicated Keyboard, Cruise 
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Figure D-1 6. Channel Activity Summary — Dedicated Keyboard, Descent 
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Figure D-17 . Workload Summary— Multifunction Versus Dedicated Keyboard, Cruise 
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Figure D-18. Workload Summary— Multifunction Versus Dedicated Keyboard , Descent 
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